Bayesian Versus Orthodox Statistics: Which Side Are You On?

“For a p value, if the null is true, any value in the interval 0 to 1 is equally likely no matter how much data you collect (Rouder et al., 2009).”

“For this reason, sooner or later, you are guaranteed to get a significant result if you run subjects long enough and stop when you get the p value you want (e.g., Wagenmakers, 2007).”

“The Bayes Factor (BF10) provides the likelihood ratio of the experimental hypothesis being true compared to the null hypothesis being true (Dienes, 2011).”

“…(0.08), but the Bayesian factor was 0.567, i.e. inconclusive (a Bayes factor > 3 is considered as substantial evidence for the alternative hypothesis [H1], values < 0.333 indicate substantial evidence for the null [H0], and values between 0.333 and 3 indicate a lack of sensitivity; Dienes, 2011).”

“Partly, this interest is driven by its increased accessibility: computationally expensive fitting procedures used in Bayesian statistics can now be run onmany personal computers [6, 27].”

“Bayes factors express the probability of the data given H0 relative to H1 (Dienes, 2011).”

“Bayesian statistics quantifies the evidence for the null and alternative hypotheses by providing information about the probability of each of the hypotheses being true, given the data at hand (Chambers, 2019; Dienes, 2011; Dienes & McLatchie, 2018; Gelman & Loken, 2014; Ortega & Navarrete, 2017; Simonsohn, 2015; Wagen…”

“…hypotheses, where criteria for a significant p-value have been fulfilled, they do not allow for valid inferences to be made on the acceptance of the null hypothesis (in light of established non-significant differences between conditions: Wagenmakers, 2007; Kruschke, 2010; Dienes, 2011).”

“In order to test whether the null hypothesis is supported, rather than there being insufficient evidence of a difference, a Bayes factor analysis was conducted (e.g. Dienes, 2011).”

“Conventionally, a Bayes factor greater than 3 is considered as substantial evidence in favor of the null or the alternative hypothesis (Jeffreys, 1961; Dienes, 2011).”

“To assess the evidence that decoding performance or parameter values differed from chance, we calculated Bayes factors (Dienes, 2011; Jeffreys, 1961; Kass and Raftery, 1995; Rouder et al., 2009; Wagenmakers, 2007).”

“Thus, we modeled an expectation for unconscious knowledge with a half-normal with a mode of zero and a standard deviation of 5% (following the recommendations of [45], Appendix).”

“Bayes Factors offer the only solution, because they express the strength of evidence for one theory over another, something that significance testing does not allow [69].”

“…using JZS Bayes factor (BF) tests with default prior scales (Rouder et al., 2012) so that a BF 0.33 implies substantial evidence for the null hypothesis, 0.33 < B < 3 suggests insensitivity of the data, and B > 3 implies substantial evidence for the alternative hypothesis (see Dienes, 2011).”

“Bayesian analyses compare the probabilities for both the null and experimental hypotheses, thereby reducing the likelihood of a Type-I error (Dienes, 2011).”

“Third, the Bayes factor usually obeys the likelihood principle unless the prior depends on sample size (refer to Dienes, 2011 for more details).”

“Smoking rates tend to be higher in those who suffer from chronic pain compared with the general population (Ditre et al., 2011), yet few studies have examined whether chronic pain is prospectively associated with smoking initiation; this merits further investigation.”

“In contrast, a BF10 <0.33 is regarded as moderate evidence for the null hypothesis (with BF10 <0.1 indicating strong evidence), and a BF10 between 1 and 0.33 as anecdotal evidence (Dienes, 2011; Nuzzo, 2017).”

“We will adjust for multiple contrasts using a Holm-modified Bonferroni method but recognize Bayesian caution against doing so and the need for sensitivity analyses both with and without them [85, 86].”

“As frequentist statistics cannot provide evidence in favour of the null hypothesis [61, 62], we ran a Bayesian ANCOVA on behavioural intentions, using JASP [63].”

“Bayes factors indicate how strongly the data supports a hypothesis and can be used in the case of non-significant results to determine whether this is merely due to a lack of power (Dienes, 2011).”

“Simply put, strict “null hypothesis significance testing” cannot distinguish whether there is a true null effect or whether the data are insensitive [9].”

“32 BF s are expressed in terms of the relative strength of evidence when comparing two hypotheses, given the data and prior expectations about the parameters.”

“The Bayes factor indicates evidence for the null hypothesis if sufficiently low (around 1/3 or lower) [30,31].”

“It expresses the relative probability of the data under, e.g., the null vs. the alternative hypothesis (Gallistel, 2009; Rouder et al., 2009; Dienes, 2011; Morey, 2015).”

“…has become increasingly apparent that the Bayesian approach to data analysis comes with considerable advantages over classical statistics, both theoretical and practical (e.g. [Die11]; for a systematic overview of more than 1500 articles reporting Bayesian analyses in psychology see [vdSWR ∗ 17]).”

“By implementing these Bayesian statistics we were justified in continuing to collect data until either substantial support for the hypothesis was obtained (BFmain 3) or, conversely, substantial support for the null was obtained (BFmain 1/3, Jeffreys 1961; Dienes 2011).”

“, 2018 for a review), and the problem of multiple testing does not matter within a Bayesian analysis (Dienes, 2011). Our Bayesian two-way ANOVAs were based on Wetzels et al. (2012), and we interpreted the Bayes factors according to Wagenmakers et al.”

“Likelihood statistics also differ from other statistical paradigms (eg, Bayesian statistics) wherein a strong prior belief (ie, prior probability) about a study hypothesis, combined with the likelihood of the data, produces an “updated” belief.(120,121) For example, if a clinician has very strong personal rationale fo…”

“These were calculated using the procedure recommended by Dienes (2011), implemented within an R script by Baguley and Kaye (2010).”

“The Bayes factor states the ratio between the evidence supporting the hypothesis relative to the null hypothesis (Dienes, 2011), such that a Bayes factor with a value of less than 1/3 indicates support for the null hypothesis.”

“dence for an effect (Dienes, 2011; Rouder & Morey, 2011; Wagenmakers, 2007), and this particularly as sample size”

“Use of such analyses in this context allows for a direct comparison between competing models of the effects (e.g. Dienes 2011, 2014; Wagenmakers et al.”

“Relative to frequentist statistics, the Bayesian approach has the advantage of relying on a model comparison rationale and of adopting a model selection strategy to quantify the strength of evidence associated with as well as against each model as a predictor of vocabulary abilities (Dienes, 2011; Morey & Rouder, 2011…”

“Throughout many of our results, we also employed Bayesian analyses to supplement our null hypothesis tests (Dienes, 2011).”

“Although avoiding the .05 criterion problem, Bayesian inference, using the Bayes factor (i.e., Dienes, 2011; Goodman, 1999; Rouder, Speckman, Sun, Morey, & Iverson, 2009) still remains susceptible to distortion through aggregation or averaging data across multiple trials.”

“Thus, to represent the plausibility of different possible population effects, we used a half-normal with a standard deviation of 5% (following the recommendation of Dienes, 2011, Appendix).”

“…role of Bayesian versus frequentist approaches, and the consequences of each in terms of accurate representation of prior assumptions, the influence of biases due to those prior assumptions, and desired goals in making statistical inferences (e.g., Clarke & Gustafson, 1998; Dienes, 2011).”

“In psychology, Bayesian inference and frequentist hypothesis testing have been compared by e.g. Dienes (2011); recommendations for their respective usage are provided by statisticians (e.g., Little, 2006; Efron, 2005).”

“BFs larger than 1 indicate that there is more evidence for the alternative hypothesis than the null hypothesis (Dienes, 2011) with a BF >3 considered as “ substantial ” evidence for the alternative hypothesis, i.e., decoding above-or below-chance level.”

“…science research more broadly), the literature has emphasized its structural and interpretative problems (e.g., Rozeboom, 1960; Cohen, 1994; McLean & Ernest, 1998; Dienes, 2011) with a lamentable influence on normative practice (Gigerenzer & Sedlmeier, 1989; Halk & Greenbaum, 1995, Lecoutre, 2006).”

“Bayes factors were used for statistical testing, as both the presence and the absence of the effects are of theoretical interest (Rouder et al. 2009; Dienes 2011; Wetzels et al. 2011).”

“However, evidence indicates that many people misinterpret frequentist estimates as if they were Bayesian (Dienes, 2011) and so changing outlook on probability is less likely to be a problem.”

“To interpret nonsignificant results, we computed Bayes factors, which quantify whether a nonsignificant result supported the null hypothesis or whether the data were just insensitive (Dienes, 2011).”

“To determine whether decoding accuracy was above chance and to quantify differences in decoding accuracies across conditions, Bayes factors (BF) were computed using the R package BayesFactor ( Dienes, 2011; Rouder et al., 2009; Teichmann, 2022).”

“Subsequently, we used a half normal with SD # 19.16 (see Dienes, 2011).”

“As drawing a definite conclusion from a nonsignificant result is problematic (e.g., Dienes, 2011, p. 282), it is more worthwhile to highlight the exemplary purpose of varying expected value.”

“Contrary to frequentist statistics, this allows for conclusions about the evidence in support of H0 (Dienes, 2011, 2014).”

“evidence for the alternative hypothesis over the null hypothesis [82].”

“BFs indicate the relative evidence for two competing theories on the basis of the collected data (Dienes, 2011).”

“Bayes factors can be used for the evaluation of multiple hypotheses without the need to apply multiple comparisons corrections [44 – 46], thus we only applied multiple corrections comparisons for purely frequentist analyses (i.e., the correlations, see Results and Supplementary).”

“Nothing at all follows from a nonsignificant result in itself, but a Bayes factor (B) can indicate substantial evidence for the null hypothesis (B < 1/3), that the data are insensitive (1/3< B< 3), or substantial evidence for the alternative (B> 3; Dienes, 2011, 2014; Fu et al., 2016).”

“A power analysis using G∗Power 3.1 (Faul et al., 2007), with a power of 0.80 and an alpha of 0.05, showed that a total of 128 participants were enough to detect a medium effect size (f = 0.25) in the 2 (Number of critical lures per list at test) × 2 (BAS) interaction of our interest.”

“…combines means and medians: we primarily use means for ratings to provide an overall measure of central tendency, but in cases where a participant ’ s rating is considered an outlier, we replace that specific outlier with the participant’s median rating to mitigate the impact of extreme values.”

“…over 3 can be considered substantial support for the experimental hypotheses; values less than 1/3 can be considered substantial evidence for the null and values around 1 indicate no strong support in either direction, suggesting data insensitivity (see Dienes, 2008, 2011, 2014 for rationale).”

“(1963) introduced Bayesian analysis to psychology, and more recently a number of researchers (Dienes 2011, Kruschke 2010a, Kruschke 2010b, Lee & Wagenmakers 2005, Wagenmakers 2007) have advocated that Bayesian statistics should replace the Null Hypothesis Statistical Testing (NHST) paradigm, which is still the most po…”