Bayesian Versus Orthodox Statistics: Which Side Are You On?

“For a p value, if the null is true, any value in the interval 0 to 1 is equally likely no matter how much data you collect (Rouder et al., 2009).”

“For this reason, sooner or later, you are guaranteed to get a significant result if you run subjects long enough and stop when you get the p value you want (e.g., Wagenmakers, 2007).”

“This decision was motivated by several factors, some of which are philosophical and beyond the scope of the present article (for a few examples see Dienes, 2011; Gelman & Hill, 2007; Hoekstra et al., 2014; Kruschke, 2011, 2014).”

“To do so, we used the Bayes factor (B; http://www.lifesci.sussex.ac.uk/home/ Zoltan_Dienes/inference/-bayes_factor.swf; Capozzi et al., 2016; Dienes, 2011, 2014; Rothkirch et al., 2015), which indicates the relative strength of evidence for the alternative hypothesis over the null, with values less…”

“Because we were particularly interested in potential null effects (i.e., no costs to uncued items in WM), we used Bayes Factors (BF; Dienes, 2011; Kass & Raftery, 2012) calculated from Bayesian t tests and Bayesian ANOVAs (Rouder, Morey, Speckman, & Province, 2012) to establish whether the lack of…”

“The Bayes Factor for the interaction term was 0.053, which is ‘‘substantial” evidence in favor of the null hypothesis (Dienes, 2011, 2014). consistent with the analysis above.”

“, [1-3]).”

“To account for this limitation, we additionally calculated Bayes factors (24) to probe whether the two counterparts of our ROIs were similarly active (null hypothesis) in the maintenance phases of the memory tasks or not (alternative hypothesis) (also see Materials and Methods).”

“This may highlight the concerns regarding using only null-hypothesis significance testing for scientific inference raised repeatedly in the literature (Dienes 2011).”

“Bayes factors > 3 suggest ‘substantial’ evidence for the alternative hypothesis and Bayes factors < 0.33 indicate ‘substantial’ evidence for the null hypothesis (Dienes, 2011, 2014).”

“Bayes factors (BFs) are a powerful tool for evaluating evidence both for the research hypothesis and for the null hypothesis (e.g., Rouder et al., 2009; Dienes, 2011; Kruschke, 2011).”

“We used Bayes factors to interpret the evidential value of nonsignificant findings (Dienes, 2011, 2014; Rouder, Speckman, Sun, Morey, & Iverson, 2009).”

“A BF10 greater than 3 is typically suggestive of substantial evidence in favor of the alternative hypothesis (Jeffreys 1961; Dienes 2011). fMRI Data Acquisition Images were collected with a Philips Achieva 3T MRI scanner (Philips Medical Systems).”

“Bayes factors have gained some traction in psychology as an alternative hypothesis-testing framework (e.g., Rouder et al., 2009; Dienes, 2011; Kruschke, 2011).”

“that suffices for the study of conscious versus unconscious processing (Dienes & Seth, 2010; Reingold & Merikle, 1988). Alternatively, stricter measures of awareness can be used in future studies, controlling not only for the visibility of the feature of interest but for the visibility of lower-level features as well,…”

“To compute the Bayes factor, we hypothesized that the unimodal sensory signals would combine following a half-normal distribution [45,46].”

“Using the Bayesian approach described by Dienes ((2011); see also http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/infer ence/Bayes.htm), we compared H0 with two alternate hypotheses. Alternate hypothesis H1 was that the plausible size of the above interaction term for the population of young and older ages consider…”

“In addition, a comparable Bayesian analysis was conducted to ascertain the strength of support for the experimental hypothesis (Dienes, 2011).”

“With Bayesian hypothesis testing, we could directly evaluate the relative strength of evidence for the null and alternative hypotheses, providing quantification of the degree to which the data support either hypothesis (Dienes, 2011, Wagenmakers et al., 2017).”

“…some Bayesian statisticians (e.g., Lindley, 2000) and advocates of the Bayesian way in psychology, who at University Libraries | Virginia Tech on November 1, 2016epm.sagepub.comDownloaded from argue for the uptake of Bayesian rationality across the board (e.g., Dienes, 2011), should be resisted.”

“With Bayesian hypothesis testing, we could directly evaluate the relative strength of evidence for the null and alternative hypotheses, providing quantification of the degree to which the data support either hypothesis (Dienes, 2011; Wagenmakers et al., 2018).”

“In addition, considering both doubts about the reliability of reports in type 2 blindsight as well as the inconsistent results of previous studies, it appears necessary to test how robust the effect of confidence vs. visibility occurs.”

“Adopting the procedure advocated by Dienes (2011), the alternative hypothesis was d′1 following a half-normal distribution with a standard deviation equivalent to the d′1 observed for participants showing above-chance decision accuracy in the first 75% of trials (0.38).”

“9 (Team, 2018) to interpret all null effects: BF > 3 indicates evidence for experimental hypothesis, and BF < 1/3 indicates evidence for the null hypothesis (Dienes, 2011; Wagenmakers et al., 2017)).”

“First, p - values are defined as long term frequencies of an event happening, meaning that any change in the sampling procedure after the beginning of data collection (e.g., increasing the sample size) will lead to invalid results (Dienes, 2011; Kruschke, 2014).”

“Dienes (2011) also suggests estimating population standard deviation value by taking half of the population mean, yielding .”

“COGNITION AND EMOTION, 2015, 29 (8) 1429 et al., 2011; White, Ratcliff, Vasey, & McKoon, 2010a, 2010b).”

“A correction for multiple comparisons is not necessary for Bayesian t-tests (Dienes, 2011).”

“…in addition to avoiding some of the problems of null hypothesis significance testing, these analyses are able to estimate the relative strength of evidence for and against null and alternative hypotheses (Dienes, 2011; Kruschke, 2010, 2011; Wagenmakers, Wetzels, Borsboom, & van der Maas, 2011).”

“Dienes, Z. (2011). Bayesian versus orthodox statistics: Which side are”

“…alternative hypothesis is favoured over the null hypothesis: larger BF10 values argue in favour of the alternative hypothesis, with values above 3 denoting “substantial evidence” in favour of the alternative hypothesis; by contrast, values less than 1 favour the null hypothesis; see Dienes (2011).”

“Authors such as Dienes (2011); Wetzels et al. (2011), and Kruschke (2011) suggest Bayesian inference is more coherent than NHST, Bayes factors can actually provide evidence in favor of the null hypothesis, and Bayesian inference is much richer than NHST.”

“We report the Bayes factor B10, which represents howmuch of the variance of themetric is explained by a parameter [if B10 < 1/3, then there is no impact; if B10 > 3, then the impact is strong; and if 1/3 < B10 < 3, then the results are inconclusive (39, 40)].”

“To see this, we again use Bayes’ rule and obtain”

“33 and 3 as being insensitive in distinguishing between the two (Dienes, 2011; Jeffreys, 1961).”

“As a result of this crisis, psychological and neuroscientific sciences are gradually transitioning toward new practices and norms, with respect to both the way studies are designed and predefined and the way the data are analyzed and evaluated (Cumming, 2014; Dienes, 2011).”

“Bayes factor values of less than 0.33 indicate substantial support for the null hypothesis and values greater than 3 indicate substantial support for the alternative (e.g., Dienes, 2011).”

“To determine whether there was evidence for no difference between the RTs of the two conditions, a Bayes factor (Dienes, 2011) was calculated using Dienes’s online calculator (www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inference/ bayes_factor.swf).”

“Recent developments in the literature on methods in the field of psychology strongly favor Bayesian analyses over the more commonly employed nullhypothesis testing (NHST) approach (Wagenmakers, Lodewyckx, Kuriyal, & Grasman, 2010; Kruschke, 2010b; Dienes, 2011; Kruschke, 2013).”

“A Bayes factor of one indicates that the evidence is exactly neutral with respect to the experimental and null hypotheses (Dienes, 2011).”

“To further test this account, we calculated the Bayes factor (15, 16) comparing the alternative hypothesis that the geometrically appropriate corners were chosen more often than the inappropriate corners to the null hypothesis that the geometrically appropriate and inappropriate corners were chosen equally often.”

“Because we have evidence in this experiment that the inversion effect for new Thatcherized faces is not different from that in our scrambled faces, we were able to perform a Bayesian analysis of both Experiments 1 and 2 using the procedures outlined by Dienes (2011). This used the effect in the scrambled faces as the…”

“Bayesian hypothesis testing offers a much better approach (Dienes, 2011) because it seems less likely to generate false positives. psychologists for papers eventually published in Psychological Science revealed all researchers deviated from their registered plans, but that the authors of only one…”

“We therefore designed our own Bayes Factor calculator for binary data.”

“…when, for example, two researchers with the exact same data set would reach different conclusions only because they had different hypotheses (Dienes, 2011); or if one of them, by being more ambitious and performing additional tests (maybe useful control tests), would not reach…”

“Two features of the present experiments motivated us to adopt a fully Bayesian approach in handling our results (for further discussion, see Dienes, 2011; Fawcett, Lawrence, & Taylor, 2016).”

“A Bayes Factor (BF01) of 1–3 indicates anecdotal evidence, whereas a BF01 of 3–10 indicates substantial evidence for the null hypothesis (i.e., the data are 3–10 times more likely to support the null vs. alternative hypothesis) (Dienes, 2011).”

“In the Bayesian framework, the hypothesis testing is conducted using the Bayes factor, which compares and quantifies the evidence in favor or against the null/alternative hypothesis (Kruschke, 2011; Dienes, 2011). In addition, the Bayesian Credible Intervals (BCI) can be obtained based on percentiles of the posterior…”

“Substantial evidence in favor of the alternative hypothesis is typically suggested by a BF10 greater than 3 (Jeffreys 1961; Dienes, 2011).”

“As Bayesian hypothesis testing quantifies both the confidence that can be placed in the hypothesis, given the data, and its respective null, these tests do not suffer from the concerns of multiple comparisons and violation of stopping rules in frequentist statistics [32,61].”

“…Biel and Friedrich (2018), Bayes factors (BF), a useful tool for evaluating evidence both for the research hypothesis and for the null hypothesis (Dienes, 2011; Kruschke, 2011), are reported for repeated measures ANOVAs using JASP (Jeffreys’s Amazing Statistics Program, Marsman and Wagenmakers,…”

“The predictions of the theory was represented by a half-normal with a standard deviation of 1.5%, i.e. the theory allowed effects between 0 and about 3%; see [124–125] for the technique.”

“distribution is more uncertain or wider (Dienes, 2011).”

“The Bayes analysis yielded a BF of 0.10, that is, substantial evidence for the null hypothesis of chance performance [Dienes, 2011].”

“62 times in favor of the null, which is approaching a level of 3, generally considered to be substantial evidence in favor of the null (Dienes, 2011).”

“Following Dienes [32], we used a half-normal distribution with a standard deviation of.42, which corresponds to the numerical difference in betting scores between stop and double-response groups in Experiment 3 of Verbruggen et al. [18].”

“Null-hypothesis (H0) significance tests and their accompanied p-values have several shortcomings and more reliable alternative approaches, such as BF10, have been suggested (Gallistel, 2009; Dienes, 2011; Nuzzo, 2014).”

“Hence, we modelled the prediction of our hypothesis as a half-normal distribution with a mode of 0 and a standard deviation of 0.5, as suggested by Dienes, 2011 [21].”

“Since a non-significant result of a one-sample t-test cannot provide conclusive evidence for the null hypothesis, we further conducted a Bayesian analysis (Dienes, 2011).”

“To illustrate this, Dienes [12] provides a simple and amusing example summarised below: P(dying within 2 years | head bitten off by shark) = 1 Everyone who has their head bitten off by a shark will be dead 2 years later.”

“For a conceptual comparison of Bayesian vs. null hypothesis significance testing, see Dienes (2011), and for computational details, see Rouder et al. (2009).”

“That is, no conclusion follows automatically from a statistically nonsignificant result (Dienes, 2011, 2014).”

“The expected difference was modeled as a two-tailed normal distribution with a standard deviation equal to half the mean [Dienes, 2011]. r 3565 r”

“…Borsboom, & van der 功，如 (Zhu, Chen, Hu, & Zhang, 2017) 。贝叶斯统计的工具迅速发展，如 WinBUGs (Lunn, Spiegelhalter, Thomas, & Best, 2009) 、 JAGS (Plummer, 2003) 、 Stan (Carpenter et al., 2017) 和 python 语言的工具包 PyMC3 （ http://docs.pymc.io/index.html ）等。这些软件和工具包的出现， 促进贝叶斯方法在各个研究领域中的使用 (Depaoli & van de Schoot,…”

“The output of Bayesian hypothesis testing is a Bayes factor (B), which, when greater than 3 indicates that the data provides substantial support for the hypothesis and when below 1/3 indicates that the data supports the null [38,39].”

“…in (A) across task conditions (i.e., first and third targets in trial runs starting either with a switch or a repeat cue). by gathering evidence in favor of it, and thus, the alternative hypothesis can only be falsified by accepting the null hypothesis over it (Dienes, 2011; Kruschke, 2013).”

“The Bayes Factor (B10) provides the likelihood ratio of the experimental hypothesis being true compared to the null hypothesis being true (Dienes, 2011).”

“…to characterize evidence arising from the data as either supporting the presence (alternative hypothesis) or absence (null hypothesis) of an effect(Dienes, 2011; Rouder, Speckman, Sun, Morey, & Iverson, 2009; Teichmann, Moerel, Baker, & Grootswagers, 2022; Wagenmakers, 2007; Jeffreys, 1998).”

“Bayes factors (B) were calculated according to the method described by Dienes (29) to assess whether the difference in CVwp values between each acquisition technique in each voxel is better explained by the null hypothesis (a difference of 0%) or by the alternative hypothesis (a plausible difference in population mean…”