Pre-post effect sizes should be avoided in meta-analyses
Open accessAbstract
Aims: The standardised mean difference (SMD) is one of the most used effect sizes to indicate the effects of treatments. It indicates the difference between a treatment and comparison group after treatment has ended, in terms of standard deviations. Some meta-analyses, including several highly cited and influential ones, use the pre-post SMD, indicating the difference between baseline and post-test within one (treatment group). Methods: In this paper, we argue that these pre-post SMDs should be avoided in meta-analyses and we describe the arguments why pre-post SMDs can result in biased outcomes. Results: One important reason why pre-post SMDs should be avoided is that the scores on baseline and post-test are not independent of each other. The value for the correlation should be used in the calculation of the SMD, while this value is typically not known. We used data from an ‘individual patient data’ meta-analysis of trials comparing cognitive behaviour therapy and anti-depressive medication, to show that this problem can lead to considerable errors in the estimation of the SMDs. Another even more important reason why pre-post SMDs should be avoided in meta-analyses is that they are influenced by natural processes and characteristics of the patients and settings, and these cannot be discerned from the effects of the intervention. Between-group SMDs are much better because they control for such variables and these variables only affect the between group SMD when they are related to the effects of the intervention. Conclusions: We conclude that pre-post SMDs should be avoided in meta-analyses as using them probably results in biased outcomes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Single-arm pre-post studies may constitute a low quality with the perceived overestimated effect size compared to randomised trials.
“Furthermore, single-arm pre-post studies may constitute a low quality with the perceived overestimated effect size compared to randomised trials.42”
From Mindfulness-based interventions for psychological wellbeing and quality of life in men with prostate cancer: A systematic review and meta-analysis (2021) · cites this paper as [42]
In “Pre-post SMDs do not give reliable information about treatment effects” (page unavailable), the paper says uncontrolled pre-post changes can reflect natural recovery rather than treatment, and that open studies and randomised trials may not be comparable. It supports the concern about the reliability of single-arm pre-post estimates, but does not establish that their effect sizes are systematically overestimated relative to randomised trials. (LLM, conf=0.84)
Experimental — accuracy not yet measuredBetween-group SMDs provide better control over covariates compared with within-group SMDs.
“Furthermore, between-group SMDs provide better control over covariates compared with within-group SMDs.41”
From Drug Efficacy in the Treatment of Antipsychotic-Induced Akathisia: A Systematic Review and Network Meta-Analysis (2024) · cites this paper as [41]
Pre-post standardized mean differences were summarized in a narrative synthesis because it is not recommended to analyze them within a meta-analysis.
“As it is not recommended to analyze pre-post standardized mean differences within a metaanalysis (Cuijpers et al., 2017), the findings of pre-post studies were summarized in a narrative synthesis.”
From A Systematic Review and Meta-Analysis Exploring Effects of Third-Wave Psychological Therapies on Hearing-Related Distress, Depression, Anxiety, and Quality of Life in People With Audiological Problems. (2022) · cites this paper as Cuijpers et al., 2017
Pre-post standardized mean differences should be avoided because they can contribute to biased outcomes and do not give reliable information about treatment effects.
“Cuijpers et al. (2017, p. 367) argued that “insofar as possible pre-post SMDs (standardized mean differences) should be avoided because they can contribute to biased outcomes and do not give reliable information about treatment effects”.”
From The effect of internet-delivered cognitive behavioral therapy for depression and anxiety on quality of life: A meta-analysis of randomized controlled trials (2023) · cites this paper as Cuijpers et al. (2017)
Pre-post SMDs are uncontrolled and it is impossible to disentangle which proportion of the SMD is caused by the intervention and which by natural recovery and other processes.
“What is even more important with regard to pre-post SMD, is that “it only calculates the change within one group. That means that the pre-post SMD is uncontrolled and it is impossible to disentangle which proportion of the SMD is caused by the intervention and which by natural recovery and other processes” (Cuijpers et al., 2017, p. 365).”
From The effect of internet-delivered cognitive behavioral therapy for depression and anxiety on quality of life: A meta-analysis of randomized controlled trials (2023) · cites this paper as Cuijpers et al., 2017
Sensitivity analyses for correlations set to 0.00, 0.75, and 0.99 were conducted to examine the robustness of the findings.
“We also conducted sensitivity analyses for correlations set to 0.00, 0.75, and 0.99 to examine the robustness of our findings [42].”
Most patients favor psychotherapy over pharmacotherapy.
“most patients favor psychotherapyoverpharmacotherapy.”
From Long-term Outcomes of Cognitive Behavioral Therapy for Anxiety-Related Disorders: A Systematic Review and Meta-analysis (2020) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032 · see citing works
Hamilton, M. (1960) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.23.1.56 · Cited by 32,817 · see citing works
Selective Cortical Undercutting As a Means of Modifying and Studying Frontal Lobe Function in Man
Scoville, W. B. (1949) · Journal of neurosurgery · doi:10.3171/jns.1949.6.1.0065 · Cited by 7,373 · see citing works
Kirsch, I., Deacon, B. J., Huedo-Medina, T. B., Scoboria, A., Moore, T. J., & Johnson, B. T. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050045 · Cited by 2,442 · see citing works
Cuijpers, P., Karyotaki, E., Weitz, E., Andersson, G., Hollon, S. D., & van Straten, A. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.02.026 · Cited by 586 · see citing works
Self-citationHesser, H., Weise, C., Westin, V. Z., & Andersson, G. (2011) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.12.006 · Cited by 340 · cited 1× · see citing works
“However, many authors use a fixed value that is not based on any empirical data, for example the value of r = 0.75 or r = 0.7 is used in many studies (Hesser et al. 2011; Hofmann et al. 2014; Johnsen & Friborg, 2015).”
MethodInfluentialJohnsen, T. J., & Friborg, O. (2015) · Psychological Bulletin · doi:10.1037/bul0000015 · Cited by 331 · see citing works
Effect of cognitive-behavioral therapy for anxiety disorders on quality of life: A meta-analysis.
Hofmann, S. G., Wu, J. Q., & Boettcher, H. (2014) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0035491 · Cited by 191 · see citing works
Weitz, E. S., Hollon, S. D., Twisk, J., van Straten, A., Huibers, M. J. H., David, D., DeRubeis, R. J., Dimidjian, S., Dunlop, B. W., Cristea, I. A., et al. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2015.1516 · Cited by 187 · see citing works
Self-citationCuijpers, P., Weitz, E., Twisk, J., Kuehner, C., Cristea, I., David, D., DeRubeis, R. J., Dimidjian, S., Dunlop, B. W., Faramarzi, M., et al. (2014) · Depression and Anxiety · doi:10.1002/da.22328 · Cited by 158 · see citing works
Self-citationDivergent outcomes in cognitive-behavioral therapy and pharmacotherapy for adult depression
Vittengl, J. R., Jarrett, R. B., Weitz, E., Hollon, S. D., Twisk, J., Cristea, I., David, D., DeRubeis, R. J., Dimidjian, S., Dunlop, B. W., et al. (2016) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2015.15040492 · Cited by 97 · cited 1× · see citing works
“The details on the identification, inclusion and data extraction have been reported elsewhere (Cuijpers et al. 2014b; Weitz et al. 2015; Vittengl et al. 2016).”
MethodSelf-citationEffectiveness of Long-term Psychodynamic Psychotherapy: A Meta-analysis
Leichsenring, F. (2008) · JAMA · doi:10.1001/jama.300.13.1551 · Cited by 65 · cited 2× · see citing works
“Another example is a meta-analysis in the Journal of the American Medical Association, showing that long-term psychodynamic psychotherapy is effective in the treatment of mental disorders (Leichsenring & Rabung, 2008).”
BackgroundMethodvan der Lem, R., van der Wee, N. J. A., van Veen, T., & Zitman, F. G. (2012) · Psychotherapy and Psychosomatics · doi:10.1159/000330890 · Cited by 63 · see citing works
van der Lem, R., de Wever, W. W., van der Wee, N. J., van Veen, T., Cuijpers, P., & Zitman, F. G. (2012) · BMC Psychiatry · doi:10.1186/1471-244x-12-192 · Cited by 37 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- Beck, 1996, BDI-II. Beck Depression Inventory
- Higgins, 2011, Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]
Cited by
Sorted by citations
Farooq, A., Martin, A., Janssen, X., Wilson, M. G., Gibson, A., Hughes, A. R., & Reilly, J. J. (2019) · Obesity Reviews · doi:10.1111/obr.12953 · Cited by 469 · see citing works
Etzelmueller, A., Vis, C., Karyotaki, E., Baumeister, H., Titov, N., Berking, M., Cuijpers, P., Riper, H., & Ebert, D. D. (2020) · Journal of Medical Internet Research · doi:10.2196/18100 · Cited by 356 · see citing works
Self-citationvan Dis, E. A. M., van Veen, S. C., Hagenaars, M. A., Batelaan, N. M., Bockting, C. L. H., van den Heuvel, R. M., Cuijpers, P., & Engelhard, I. M. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.3986 · Cited by 335 · see citing works
Self-citationGál, É., Ștefan, S., & Cristea, I. A. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.09.134 · Cited by 289 · see citing works
Self-citationCHASTIN, S. F. M., Abaraogu, U. O., Bourgois, J., Dall, P., Darnborough, J., Duncan, E., Dumortier, J., Jiménez‐Pavón, D., McParland, J. L., Roberts, N., et al. (2021) · Sports Medicine · doi:10.1007/s40279-021-01466-1 · Cited by 315 · see citing works
Effects of virtual reality on learning outcomes in K-6 education: A meta-analysis
Villena-Taranilla, R., Tirado‐Olivares, S., Gutiérrez, R. C., & González‐Calero, J. A. (2022) · Educational Research Review · doi:10.1016/j.edurev.2022.100434 · Cited by 350 · see citing works
Beyond small, medium, or large: points of consideration when interpreting effect sizes
Bakker, A., Cai, J., English, L., Kaiser, G., Mesa, V., & Van Dooren, W. (2019) · Educational Studies in Mathematics · doi:10.1007/s10649-019-09908-4 · Cited by 302 · see citing works
Villinger, K., Wahl, D. R., Boeing, H., Schupp, H. T., & Renner, B. (2019) · Obesity Reviews · doi:10.1111/obr.12903 · Cited by 290 · see citing works
Antonelli, M., Barbieri, G., & Donelli, D. (2019) · International Journal of Biometeorology · doi:10.1007/s00484-019-01717-x · Cited by 271 · see citing works
Reid, J. E., Laws, K. R., Drummond, L., Vismara, M., Grancini, B., Mpavaenda, D., & Fineberg, N. A. (2021) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2021.152223 · Cited by 266 · see citing works
Murray, S. B., Quintana, D. S., Loeb, K. L., Griffiths, S., & Le Grange, D. (2019) · Psychological Medicine · doi:10.1017/s0033291718002088 · Cited by 258 · see citing works
Türk, F., & Waller, G. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101856 · Cited by 257 · see citing works
Wijnia, L., Noordzij, G., Arends, L. R., Rikers, R. M. J. P., & Loyens, S. M. M. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09864-3 · Cited by 455 · see citing works
Linardon, J., Fairburn, C. G., Fitzsimmons‐Craft, E. E., Wilfley, D. E., & Brennan, L. (2017) · Clinical Psychology Review · doi:10.1016/j.cpr.2017.10.005 · Cited by 224 · see citing works
Effects of lavender on anxiety: A systematic review and meta-analysis
Donelli, D., Antonelli, M., Bellinazzi, C., Gensini, G. F., & Firenzuoli, F. (2019) · Phytomedicine · doi:10.1016/j.phymed.2019.153099 · Cited by 231 · see citing works
Statistical considerations for reporting and planning heart rate variability case‐control studies
Quintana, D. S. (2017) · Psychophysiology · doi:10.1111/psyp.12798 · Cited by 192 · see citing works
Whiteside, S. P. H., Sim, L. A., Morrow, A. S., Farah, W. H., Hilliker, D. R., Murad, M. H., & Wang, Z. (2020) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-019-00303-2 · Cited by 190 · see citing works
Karkou, V., Aithal, S., Zubala, A., & Meekums, B. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00936 · Cited by 186 · see citing works
Linardon, J. (2018) · International Journal of Eating Disorders · doi:10.1002/eat.22897 · Cited by 174 · see citing works
Karami, B., Koushki, R., Arabgol, F., Rahmani, M., & Vahabie, A. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.665326 · Cited by 174 · see citing works
Spinelli, C., Wisener, M., & Khoury, B. (2019) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2019.03.003 · Cited by 170 · see citing works
Carr, R. M., Prestwich, A., Kwaśnicka, D., Thøgersen‐Ntoumani, C., Gucciardi, D. F., Quested, E., Hall, L., & Ntoumanis, N. (2018) · Health Psychology Review · doi:10.1080/17437199.2018.1532312 · Cited by 162 · see citing works
Intranasal oxytocin, social cognition and neurodevelopmental disorders: A meta-analysis
Keech, B., Crowe, S., & Hocking, D. R. (2018) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2017.09.022 · Cited by 145 · see citing works
Augmentation therapies for treatment-resistant depression: systematic review and meta-analysis
Strawbridge, R., Carter, B., Marwood, L., Bandelow, B., Tsapekos, D., Nikolova, V. L., Taylor, R., Mantingh, T., de Angel, V., Patrick, F., et al. (2019) · British Journal of Psychiatry · doi:10.1192/bjp.2018.233 · Cited by 140 · see citing works
Hunt, C., Cordeiro, T. M. E., Suchting, R., de Dios, C., Cuellar, V. A., Soares, J. C., Dantzer, R., Teixeira, A. L., & Selvaraj, S. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.08.010 · Cited by 144 · see citing works
Linardon, J., Gleeson, J., Yap, K., Murphy, K., & Brennan, L. (2019) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2018.1517389 · Cited by 134 · see citing works
Wergeland, G. J., Riise, E. N., & Öst, L. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101918 · Cited by 136 · see citing works
Wewege, M. A., & Jones, M. D. (2020) · Journal of Pain · doi:10.1016/j.jpain.2020.04.003 · Cited by 133 · see citing works
Long-term effects of internet-supported cognitive behaviour therapy
Andersson, G., Rozental, A., Shafran, R., & Carlbring, P. (2018) · Expert Review of Neurotherapeutics · doi:10.1080/14737175.2018.1400381 · Cited by 117 · see citing works
Mojtabavi, H., Saghazadeh, A., Valenti, V., & Rezaei, N. (2020) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2020.101162 · Cited by 100 · see citing works
Singla, D. R., Lawson, A., Kohrt, B. A., Jung, J. W., Meng, Z., Ratjen, C., Zahedi, N., Dennis, C., & Patel, V. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.4556 · Cited by 136 · see citing works
Educational interventions designed to develop nurses’ cultural competence: A systematic review
Oikarainen, A., Mikkonen, K., Kenny, A. J., Tomietto, M., Tuomikoski, A., Meriläinen, M. H., Miettunen, J., & Kääriäinen, M. (2019) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2019.06.005 · Cited by 124 · see citing works
Schefft, C., Kilarski, L. L., Bschor, T., & Köhler, S. (2017) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2017.07.004 · Cited by 123 · see citing works
The Effectiveness of Automated Writing Evaluation on Writing Quality: A Meta-Analysis
Zhai, N., & Ma, X. (2022) · Journal of Educational Computing Research · doi:10.1177/07356331221127300 · Cited by 169 · see citing works
Biodiversity impacts of the 2019–2020 Australian megafires
Driscoll, D. A., Macdonald, K. J., Gibson, R. K., Doherty, T. S., Nimmo, D. G., Nolan, R. H., Ritchie, E. G., Williamson, G. J., Heard, G. W., Tasker, E. M., et al. (2024) · Nature · doi:10.1038/s41586-024-08174-6 · Cited by 121 · see citing works
Peeters, N., van Passel, B., & Krans, J. (2021) · British Journal of Clinical Psychology · doi:10.1111/bjc.12324 · Cited by 140 · see citing works
Kennedy, R. S. (2020) · Aggression and Violent Behavior · doi:10.1016/j.avb.2020.101485 · Cited by 128 · see citing works
Fondren, K., Lawson, M., Speidel, R., McDonnell, C. G., & Valentino, K. (2019) · Children and Youth Services Review · doi:10.1016/j.childyouth.2019.104691 · Cited by 127 · see citing works
Effects of mindfulness meditation interventions on depression in older adults: A meta-analysis
Reangsing, C., Rittiwong, T., & Schneider, J. K. (2020) · Aging & Mental Health · doi:10.1080/13607863.2020.1793901 · Cited by 114 · see citing works
Elbers, S., Wittink, H. M., Konings, S., Kaiser, U., Kleijnen, J., Pool, J. J. M., Köke, A. J. A., & Smeets, R. (2021) · European Journal of Pain · doi:10.1002/ejp.1875 · Cited by 108 · see citing works
Arshad, U., Farhat-ul-Ain, F.-U., Gauntlett, J., Husain, N., Chaudhry, N., & Taylor, P. J. (2019) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12583 · Cited by 106 · see citing works
Hancock, J., Liu, S. X., Luo, M., & Mieczkowski, H. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4053961 · Cited by 101 · see citing works
Functionality appreciation and its correlates: Systematic review and meta-analysis
Linardon, J., Messer, M., & Tylka, T. L. (2023) · Body Image · doi:10.1016/j.bodyim.2023.02.002 · Cited by 100 · see citing works
Man, S. S., Wen, H., & So, B. C. L. (2023) · Journal of Safety Research · doi:10.1016/j.jsr.2023.11.011 · Cited by 120 · see citing works
Antonelli, M., Donelli, D., & Fioravanti, A. (2018) · Rheumatology International · doi:10.1007/s00296-018-4081-6 · Cited by 102 · see citing works
Effects of mindfulness interventions on depressive symptoms in adolescents: A meta-analysis
Reangsing, C., Punsuwun, S., & Schneider, J. K. (2020) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2020.103848 · Cited by 100 · see citing works
Hilbert, A., Petroff, D., Herpertz, S., Pietrowsky, R., Tuschen‐Caffier, B., Vocks, S., & Schmidt, R. (2020) · International Journal of Eating Disorders · doi:10.1002/eat.23297 · Cited by 97 · see citing works
Kuzminskaite, E., Gathier, A. W., Cuijpers, P., Penninx, B., Ammerman, R., Brakemeier, E., Bruijniks, S., Carletto, S., Chakrabarty, T., Douglas, K., et al. (2022) · Lancet psychiatry · doi:10.1016/s2215-0366(22)00227-9 · Cited by 69 · see citing works
Self-citationLeung, C., Pei, J., Hudec, K., Shams, F., Munthali, R., & Vigo, D. (2022) · Journal of Medical Internet Research · doi:10.2196/36004 · Cited by 94 · see citing works
Sequí‐Domínguez, I., Álvarez‐Bueno, C., Martínez‐Vizcaíno, V., Fernández-Rodríguez, R., Saz‐Lara, A., & Cavero‐Redondo, I. (2020) · Journal of Medical Internet Research · doi:10.2196/17790 · Cited by 94 · see citing works
Scott, F., Hampsey, E., Gnanapragasam, S., Carter, B., Marwood, L., Taylor, R. W., Emre, C., Korotkova, L., Martín-Dombrowski, J., Cleare, A. J., et al. (2023) · Journal of Psychopharmacology · doi:10.1177/02698811221104058 · Cited by 87 · see citing works
Association of Music Interventions With Health-Related Quality of Life
McCrary, J. M., ALTENMÜLLER, E. O., Kretschmer, C. R., & Scholz, D. S. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.3236 · Cited by 95 · see citing works
Lee, S., Oh, J. W., Park, K., Lee, E. M., & Lee, E. M. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00800-3 · Cited by 94 · see citing works
Wei, S., Smits, M. G., Tang, X., Kuang, L., Meng, H., Ni, S., Xiao, M., & Zhou, X. (2019) · Sleep Medicine · doi:10.1016/j.sleep.2019.02.017 · Cited by 86 · see citing works
Gosmann, N. P., Costa, M. D. A., Jaeger, M. D. B., Motta, L. S., Frozi, J., Spanemberg, L., Manfro, G. G., Cuijpers, P. J., Pine, D. S., & Salum, G. A. (2021) · PLoS Medicine · doi:10.1371/journal.pmed.1003664 · Cited by 87 · see citing works
Self-citationThe effectiveness of automated writing evaluation in EFL/ESL writing: a three-level meta-analysis
Ngo, T. T.-N., Chen, H. H.-J., & Lai, K. K.-W. (2022) · Interactive Learning Environments · doi:10.1080/10494820.2022.2096642 · Cited by 108 · see citing works
Yakobi, O., Smilek, D., & Danckert, J. (2021) · Cognitive Therapy and Research · doi:10.1007/s10608-020-10177-2 · Cited by 85 · see citing works
Pantalone, D. W., Nelson, K. M., Batchelder, A. W., Chiu, C., Gunn, H. A., & Horvath, K. J. (2020) · The Journal of Sex Research · doi:10.1080/00224499.2020.1728514 · Cited by 81 · see citing works
Polihronis, C., Cloutier, P., Kaur, J., Skinner, R., & Cappelli, M. (2020) · Archives of Suicide Research · doi:10.1080/13811118.2020.1793857 · Cited by 80 · see citing works
Abbass, A., Lumley, M. A., Town, J., Holmes, H., Luyten, P., Cooper, A., Russell, L., Schubiner, H., De Meulemeester, C., & Kisely, S. (2021) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2021.110473 · Cited by 77 · see citing works
Lee, C., & Wong, J. S. (2022) · Journal of Experimental Criminology · doi:10.1007/s11292-020-09442-x · Cited by 77 · see citing works
Heterogeneity of treatment effects in trials on psychotherapy of depression.
Kaiser, T., Volkmann, C., Volkmann, A., Karyotaki, E., Cuijpers, P. J., & Brakemeier, E. (2022) · Clinical Psychology Science and Practice · doi:10.1037/cps0000079 · Cited by 77 · see citing works
Self-citationNeves, L., Correia, A. I., Castro, S. L., Martins, D., & Lima, C. F. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104777 · Cited by 76 · see citing works
Zhu, Y., Liu, Q., & Zhao, L. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13420-z · Cited by 92 · see citing works
Öst, L.-G., Enebrink, P., Finnes, A., Ghaderi, A., Havnen, A., Kvale, G., Salomonsson, S., & Wergeland, G. J. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104170 · Cited by 78 · see citing works
Perfectionism interventions targeting disordered eating: A systematic review and meta-analysis
Robinson, K., & Wade, T. D. (2021) · International Journal of Eating Disorders · doi:10.1002/eat.23483 · Cited by 86 · see citing works
A Meta-analysis of the Effectiveness of Second Language Pragmatics Instruction
Ren, W., Li, S., & Lü, X. (2022) · Applied Linguistics · doi:10.1093/applin/amac055 · Cited by 74 · see citing works
Canuto, R., Garcêz, A., Souza, R. V., Kac, G., & Olinto, M. T. A. (2020) · Obesity Reviews · doi:10.1111/obr.13143 · Cited by 71 · see citing works
Szmulewicz, A., Valerio, M. P., & Martino, D. J. (2019) · Bipolar Disorders · doi:10.1111/bdi.12841 · Cited by 68 · see citing works
Wright, M., Reitegger, F., Cela, H., Papst, A., & Gasteiger‐Klicpera, B. (2023) · Journal of Youth and Adolescence · doi:10.1007/s10964-023-01735-4 · Cited by 69 · see citing works
Espiritu, A. I., & Pasco, P. M. D. (2019) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2019.05.011 · Cited by 68 · see citing works
Effectiveness of synchronous and asynchronous online learning: a meta-analysis
Zeng, H., & Luo, J. (2023) · Interactive Learning Environments · doi:10.1080/10494820.2023.2197953 · Cited by 78 · see citing works
Harrer, M., Apolinário-Hagen, J., Fritsche, L., Salewski, C., Zarski, A.-C., Lehr, D., Baumeister, H., Cuijpers, P., & Ebert, D. D. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100374 · Cited by 63 · see citing works
Self-citationThe impact of parent-only interventions on child anxiety: A systematic review and meta-analysis
Jewell, C., Wittkowski, A., & Pratt, D. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.04.082 · Cited by 65 · see citing works
Finlay, M. J., Bridge, C. A., Greig, M. P., & Page, R. M. (2021) · Sports Medicine · doi:10.1007/s40279-021-01598-4 · Cited by 63 · see citing works
Cuijpers, P., Quero, S., Papola, D., Cristea, I. A., & Karyotaki, E. (2021) · Psychological Medicine · doi:10.1017/s0033291719003581 · Cited by 62 · see citing works
Self-citationZheng, S., Kim, H., Salzman, E., Ankenman, K., & Bent, S. (2021) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-021-04885-1 · Cited by 62 · see citing works
Allott, K., van‐der‐EL, K., Bryce, S., Parrish, E. M., McGurk, S. R., Hetrick, S. E., Bowie, C. R., Kidd, S. A., Hamilton, M. P., Killackey, E. J., et al. (2019) · Schizophrenia Bulletin · doi:10.1093/schbul/sbz134 · Cited by 62 · see citing works
Dochat, C., Wooldridge, J. S., Herbert, M. S., Lee, M. W., & Afari, N. (2021) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.03.003 · Cited by 60 · see citing works
Jugl, I., Bender, D., & Lösel, F. A. (2021) · Journal of Quantitative Criminology · doi:10.1007/s10940-021-09536-3 · Cited by 61 · see citing works
Labad, J., Montalvo, I., González-Rodríguez, A., García‐Rizo, C., Crespo‐Facorro, B., Monreal, J. A., & Palao, D. J. (2020) · Schizophrenia Research · doi:10.1016/j.schres.2020.04.031 · Cited by 61 · see citing works
The Effectiveness of College Dating Violence Prevention Programs: A Meta-Analysis
Wong, J. S., Bouchard, J., & Lee, C. (2021) · Trauma Violence & Abuse · doi:10.1177/15248380211036058 · Cited by 60 · see citing works
Abou, L., Alluri, A., Fliflet, A. M., Du, Y., & Rice, L. A. (2020) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2020.06.025 · Cited by 59 · see citing works
Robert, C., Erdt, M., Lee, J. C. B., Cao, Y., Naharudin, N. B., & Theng, Y. L. (2021) · Journal of Medical Internet Research · doi:10.2196/15649 · Cited by 58 · see citing works
Jugovac, S., O’Kearney, R., Hawes, D. J., & Pasalich, D. S. (2022) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-022-00401-8 · Cited by 55 · see citing works
Hallam, C., Simmonds‐Buckley, M., Kellett, S., Greenhill, B., & Jones, A. (2021) · Psychology and Psychotherapy: Theory, Research and Practice · doi:10.1111/papt.12286 · Cited by 54 · see citing works
Reangsing, C., Lauderman, C., & Schneider, J. K. (2022) · Journal of Integrative and Complementary Medicine · doi:10.1089/jicm.2021.0036 · Cited by 53 · see citing works
Mavilidi, M. F., Pesce, C., Benzing, V., Schmidt, M., PAAS, F. G. W. C., Okely, A. D., & Vazou, S. (2022) · Educational Research Review · doi:10.1016/j.edurev.2022.100478 · Cited by 61 · see citing works
Pfund, R., Forman, D. P., Whalen, S. K., Zech, J. M., Ginley, M., Peter, S. C., McAfee, N. W., & Whelan, J. (2023) · Addiction · doi:10.1111/add.16221 · Cited by 43 · see citing works
The effects of cognitive‐behavioral therapy for eating disorders on quality of life: A meta‐analysis
Linardon, J., & Brennan, L. (2017) · International Journal of Eating Disorders · doi:10.1002/eat.22719 · Cited by 50 · see citing works
A meta-analysis on the effectiveness of food-waste reducing nudges
Zhang, J., Huang, Y., Zhu, J., & Zhao, L. (2023) · Food Policy · doi:10.1016/j.foodpol.2023.102480 · Cited by 50 · see citing works
Mindfulness programs for problematic usage of the internet: A systematic review and meta-analysis
Fendel, J. C., Vogt, A., Brandtner, A., & Schmidt, S. (2024) · Journal of Behavioral Addictions · doi:10.1556/2006.2024.00024 · Cited by 51 · see citing works
Hwang, J., Lee, G. E., Woo, J. H., Kim, S. M., & Kwon, J. Y. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01514-4 · Cited by 50 · see citing works
Turbyne, C., Goedhart, A., de Koning, P. P., Schirmbeck, F. N., & Denys, D. A. J. P. (2021) · Frontiers in Virtual Reality · doi:10.3389/frvir.2021.657638 · Cited by 47 · see citing works
Data Literacy Training for K–12 Teachers: A Meta-Analysis of the Effects on Teacher Outcomes
Filderman, M. J., Toste, J. R., Didion, L. A., & Peng, P. (2021) · Remedial and Special Education · doi:10.1177/07419325211054208 · Cited by 46 · see citing works
Jetten, W. D., Hogenbirk, R. N. M., van Meeteren, N. L. U., Cuperus, F. J. C., Klaase, J. M., & de Jong, R. (2022) · Transplant International · doi:10.3389/ti.2022.10330 · Cited by 46 · see citing works
Vogel, J. P., Nguyen, P., Ramson, J. A., Silva, M. S. D., Pham, M. D., Sultana, S., McDonald, S. J., Adu‐Bonsaffoh, K., & McDougall, A. R. A. (2024) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2024.01.012 · Cited by 46 · see citing works
Schulz, J. M., Birmingham, T. B., Atkinson, H. F., Woehrle, E., Primeau, C. A., Lukacs, M. J., Al‐Khazraji, B. K., Khan, M. C. M., Zomar, B. O., Petrella, R. J., et al. (2019) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-100231 · Cited by 46 · see citing works
Llanos-Lagos, C., Ramírez‐Campillo, R., Moran, J., & de Villarreal, E. S. (2024) · Sports Medicine · doi:10.1007/s40279-023-01978-y · Cited by 53 · see citing works
Weng, J., Wu, X.-F., Shao, P., Liu, X.-P., & Wang, C. (2023) · Annals of Medicine · doi:10.1080/07853890.2023.2299352 · Cited by 45 · see citing works
- Authors
- Cuijpers, P., Weitz, E., Cristea, I. A., & Twisk, J.
- Affiliations
- Amsterdam Public Health (NL)
- Babeș-Bolyai University (RO)
- Corresponding author
- Cuijpers, P.
- Venue
- Epidemiology and Psychiatric Sciences
- Publisher
- Cambridge University Press (CUP)
- Published
- 2016-10-28
- Volume
- 26
- Issue
- 4
- Pages
- 364-368
- ISSN
- 2045-7960 (ISSN-L)
- 2045-7979
- PMC
- PMC6998624
- OpenAlex
- W2539720341
- Semantic Scholar
- a0bf02bc67c2e6ba75bcb6ba79dab587e691fc75
- Cited by
- 432
- References
- 14
- Citations by year
- 2026: 51 · 2025: 48 · 2024: 61 · 2023: 66 · 2022: 61 · 2021: 55 · 2020: 42 · 2019: 28 · 2018: 12 · 2017: 7 · 2016: 1
- FWCI
- 14.95
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Epidemiology and Psychiatric Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, other-oa)
- VU Research Portal (publishedVersion, other-oa)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (submittedVersion)
- Padua Research Archive (University of Padova) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- National Authority for Scientific Research and Innovation
- Autoritatea Natională pentru Cercetare Stiintifică
- Unitatea Executiva pentru Finantarea Invatamantului Superior, a Cercetarii, Dezvoltarii si Inovarii
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- standardized mean difference, effect size bias, individual patient data meta-analysis, cognitive behavioral therapy, antidepressants, depression treatment
- MeSH terms
- Antidepressive Agents, Depression, Humans, Reference Standards, Meta-Analysis as Topic, Cognitive Behavioral Therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
