Author’s reply: Letter to the Editor: Double counting due to inadequate statistics leads to false-positive findings in “Effects of creatine supplementation on memory in healthy individuals: a systematic review and meta-analysis of randomized controlled trials”

Abstract

In their Letter to the Editor, Eckert and Pascher1 correctly identified a flawed statistical analysis that warranted correction pertaining to the increased statistical power of our previous meta-analysis on creatine supplementation and memory performance.2 The inclusion of multiple memory-related outcomes from each study was accompanied with an increased number of the same participants (double counting). This could have altered the outcome estimates in the meta-analyses, indeed leading to an increased risk of false-positive findings. To address this, we have now derived the average effect size for studies presenting multiple memory performance outcomes (expressed as a composite score) through pooled means and standard deviations for each intervention group. After correction, our overall analysis showed that creatine monohydrate does not improve overall memory performance (standardized mean difference [SMD], 0.19; 95% confidence interval [CI], –0.07, 0.46; I2 = 4%; P = 0.15) (Figure 1). Subgroup analyses revealed no differences of creatine monohydrate supplementation with low (≤5 g/day; SMD, 0.25; 95%CI, –0.14, 0.64; I2 = 0%; P = 0.21) or high (>5 g/d; SMD, 0.17; 95%CI, –0.34, 0.69; I2 = 50%; P = 0.51) doses (Figure 2A). However, although young adults (aged 11–31 years) did not benefit from creatine monohydrate (SMD, 0.02; 95%CI, –0.27, 0.31; I2 = 0%, P = 0.90), its effects were prominent in older adults (aged 66–76 years; SMD, 0.80; 95%CI, 0.25–1.34; I2 = 0%; P = 0.004) (Figure 2B), which was the main finding of our previous work.2 Furthermore, no changes were displayed regarding treatment duration (≤2 weeks: SMD, 0.16 [95%CI, –0.23, 0.56], I2 = 33%, P = 0.42; >2 weeks: SMD, 0.30 [95%CI, –0.17, 0.76], I2 = 0%, P = 0.21) (Figure 2C) nor sex (women and men: SMD, 0.18 [95%CI, –0.20, 0.56, I2 = 31%, P = 0.36; women: SMD, 0.42 [95%CI, –0.14, 0.98], I2 = 0%, P = 0.14; men: SMD, –0.11 [95%CI, –1.01, 0.79], P = 0.81) (Figure 2D). Our sensitivity analyses showed no effects of creatine monohydrate on memory performance in nonencapsulated form (SMD, 0.22; 95%CI, –0.08, 0.52; I2 = 16%; P = 0.15) (Figure 3A), normal (SMD, 0.29; 95%CI, –0.09, 0.68; I2 = 35%; P = 0.14) or stressed conditions (SMD, 0.03; 95%CI, –0.46, 0.52; I2 = 0%; P = 0.91) (Figure 3B), and participants lost to follow-up (<15%) (SMD, 0.28; 95%CI, –0.02, 0.59; I2 = 1%; P = 0.07) (Figure 3C).