The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Pre-post effect sizes should be avoided in meta-analyses
Open accessAbstract
Aims: The standardised mean difference (SMD) is one of the most used effect sizes to indicate the effects of treatments. It indicates the difference between a treatment and comparison group after treatment has ended, in terms of standard deviations. Some meta-analyses, including several highly cited and influential ones, use the pre-post SMD, indicating the difference between baseline and post-test within one (treatment group). Methods: In this paper, we argue that these pre-post SMDs should be avoided in meta-analyses and we describe the arguments why pre-post SMDs can result in biased outcomes. Results: One important reason why pre-post SMDs should be avoided is that the scores on baseline and post-test are not independent of each other. The value for the correlation should be used in the calculation of the SMD, while this value is typically not known. We used data from an ‘individual patient data’ meta-analysis of trials comparing cognitive behaviour therapy and anti-depressive medication, to show that this problem can lead to considerable errors in the estimation of the SMDs. Another even more important reason why pre-post SMDs should be avoided in meta-analyses is that they are influenced by natural processes and characteristics of the patients and settings, and these cannot be discerned from the effects of the intervention. Between-group SMDs are much better because they control for such variables and these variables only affect the between group SMD when they are related to the effects of the intervention. Conclusions: We conclude that pre-post SMDs should be avoided in meta-analyses as using them probably results in biased outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, single-arm pre-post studies may constitute a low quality with the perceived overestimated effect size compared to randomised trials.42”
From Mindfulness-based interventions for psychological wellbeing and quality of life in men with prostate cancer: A systematic review and meta-analysis (2021) · cites this paper as [42]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Furthermore, between-group SMDs provide better control over covariates compared with within-group SMDs.41”
From Drug Efficacy in the Treatment of Antipsychotic-Induced Akathisia: A Systematic Review and Network Meta-Analysis (2024) · cites this paper as [41]
The cited source says what the claim attributes to it.
In the citing paper
“As it is not recommended to analyze pre-post standardized mean differences within a metaanalysis (Cuijpers et al., 2017), the findings of pre-post studies were summarized in a narrative synthesis.”
From A Systematic Review and Meta-Analysis Exploring Effects of Third-Wave Psychological Therapies on Hearing-Related Distress, Depression, Anxiety, and Quality of Life in People With Audiological Problems. (2022) · cites this paper as Cuijpers et al., 2017
The cited source says what the claim attributes to it.
In the citing paper
“Cuijpers et al. (2017, p. 367) argued that “insofar as possible pre-post SMDs (standardized mean differences) should be avoided because they can contribute to biased outcomes and do not give reliable information about treatment effects”.”
From The effect of internet-delivered cognitive behavioral therapy for depression and anxiety on quality of life: A meta-analysis of randomized controlled trials (2023) · cites this paper as Cuijpers et al. (2017)
The cited source says what the claim attributes to it.
In the citing paper
“What is even more important with regard to pre-post SMD, is that “it only calculates the change within one group. That means that the pre-post SMD is uncontrolled and it is impossible to disentangle which proportion of the SMD is caused by the intervention and which by natural recovery and other processes” (Cuijpers et al., 2017, p. 365).”
From The effect of internet-delivered cognitive behavioral therapy for depression and anxiety on quality of life: A meta-analysis of randomized controlled trials (2023) · cites this paper as Cuijpers et al., 2017
In the citing paper
“We also conducted sensitivity analyses for correlations set to 0.00, 0.75, and 0.99 to examine the robustness of our findings [42].”
In the citing paper
“most patients favor psychotherapyoverpharmacotherapy.”
From Long-term Outcomes of Cognitive Behavioral Therapy for Anxiety-Related Disorders: A Systematic Review and Meta-analysis (2020) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032
Hamilton, M. (1960) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.23.1.56 · Cited by 32,817
Selective Cortical Undercutting As a Means of Modifying and Studying Frontal Lobe Function in Man
Scoville, W. B. (1949) · Journal of neurosurgery · doi:10.3171/jns.1949.6.1.0065 · Cited by 7,374
Kirsch, I., Deacon, B. J., Huedo-Medina, T. B., Scoboria, A., Moore, T. J., & Johnson, B. T. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050045 · Cited by 2,442
Cuijpers, P., Karyotaki, E., Weitz, E., Andersson, G., Hollon, S. D., & van Straten, A. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.02.026 · Cited by 586
Self-citationHesser, H., Weise, C., Westin, V. Z., & Andersson, G. (2011) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.12.006 · Cited by 340 · cited 1×
“However, many authors use a fixed value that is not based on any empirical data, for example the value of r = 0.75 or r = 0.7 is used in many studies (Hesser et al. 2011; Hofmann et al. 2014; Johnsen & Friborg, 2015).”
MethodInfluentialJohnsen, T. J., & Friborg, O. (2015) · Psychological Bulletin · doi:10.1037/bul0000015 · Cited by 331
Effect of cognitive-behavioral therapy for anxiety disorders on quality of life: A meta-analysis.
Hofmann, S. G., Wu, J. Q., & Boettcher, H. (2014) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0035491 · Cited by 191
Weitz, E. S., Hollon, S. D., Twisk, J., van Straten, A., Huibers, M. J. H., David, D., DeRubeis, R. J., Dimidjian, S., Dunlop, B. W., Cristea, I. A., et al. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2015.1516 · Cited by 187
Self-citationCuijpers, P., Weitz, E., Twisk, J., Kuehner, C., Cristea, I., David, D., DeRubeis, R. J., Dimidjian, S., Dunlop, B. W., Faramarzi, M., et al. (2014) · Depression and Anxiety · doi:10.1002/da.22328 · Cited by 158
Self-citation
Research works printed but not linked: 2
- Beck, 1996, BDI-II. Beck Depression Inventory
- Higgins, 2011, Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]
Cited by
Sorted by citations
Farooq, A., Martin, A., Janssen, X., Wilson, M. G., Gibson, A., Hughes, A. R., & Reilly, J. J. (2019) · Obesity Reviews · doi:10.1111/obr.12953 · Cited by 469
Etzelmueller, A., Vis, C., Karyotaki, E., Baumeister, H., Titov, N., Berking, M., Cuijpers, P., Riper, H., & Ebert, D. D. (2020) · Journal of Medical Internet Research · doi:10.2196/18100 · Cited by 356
Self-citationvan Dis, E. A. M., van Veen, S. C., Hagenaars, M. A., Batelaan, N. M., Bockting, C. L. H., van den Heuvel, R. M., Cuijpers, P., & Engelhard, I. M. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.3986 · Cited by 335
Self-citationGál, É., Ștefan, S., & Cristea, I. A. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.09.134 · Cited by 289
Self-citationCHASTIN, S. F. M., Abaraogu, U. O., Bourgois, J., Dall, P., Darnborough, J., Duncan, E., Dumortier, J., Jiménez‐Pavón, D., McParland, J. L., Roberts, N., et al. (2021) · Sports Medicine · doi:10.1007/s40279-021-01466-1 · Cited by 315
Metadata
- Authors
- Cuijpers, P., Weitz, E., Cristea, I. A., & Twisk, J.
- Affiliations
- Amsterdam Public Health (NL)
- Babeș-Bolyai University (RO)
- Corresponding author
- Cuijpers, P.
- Venue
- Epidemiology and Psychiatric Sciences
- Publisher
- Cambridge University Press (CUP)
- Published
- 2016-10-28
- Volume
- 26
- Issue
- 4
- Pages
- 364-368
- ISSN
- 2045-7960 (ISSN-L)
- 2045-7979
- PMC
- PMC6998624
- OpenAlex
- W2539720341
- Semantic Scholar
- a0bf02bc67c2e6ba75bcb6ba79dab587e691fc75
- Cited by
- 432
- References
- 14
- Citations by year
- 2026: 51 · 2025: 48 · 2024: 61 · 2023: 66 · 2022: 61 · 2021: 55 · 2020: 42 · 2019: 28 · 2018: 12 · 2017: 7 · 2016: 1
- FWCI
- 14.95
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Epidemiology and Psychiatric Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, other-oa)
- VU Research Portal (publishedVersion, other-oa)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (submittedVersion)
- Padua Research Archive (University of Padova) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- National Authority for Scientific Research and Innovation
- Autoritatea Natională pentru Cercetare Stiintifică
- Unitatea Executiva pentru Finantarea Invatamantului Superior, a Cercetarii, Dezvoltarii si Inovarii
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- standardized mean difference, effect size bias, individual patient data meta-analysis, cognitive behavioral therapy, antidepressants, depression treatment
- MeSH terms
- Antidepressive Agents, Depression, Humans, Reference Standards, Meta-Analysis as Topic, Cognitive Behavioral Therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
