The sleeper effect: Artifact or phenomenon—A brief comment on Bell et al. (2013).
Abstract
OBJECTIVE: Bell, Marcus, and Goodlad (2013) recently conducted a meta-analysis of randomized controlled additive trials and found that adding an additional component to an existing treatment vis-à-vis the existing treatment produced larger effect sizes on targeted outcomes at 6-months follow-up than at termination, an effect they labeled as a sleeper effect. One of the limitations with Bell et al.'s detection of the sleeper effect was that they did not conduct a statistical test of the size of the effect at follow-up versus termination. METHOD: To statistically test if the differences of effect sizes between the additive conditions and the control conditions at follow-up differed from those at termination, we used a restricted maximum-likelihood random-effect model with known variances to conduct a multilevel longitudinal meta-analysis (k = 30). RESULTS: Although the small effects at termination detected by Bell et al. were replicated (ds = 0.17-0.23), none of the analyses of growth from termination to follow-up produced statistically significant effects (ds < 0.08; p > .20), and when asymmetry was considered using trim-and-fill procedure or the studies after 2000 were analyzed, magnitude of the sleeper effect was negligible (d = 0.00). CONCLUSION: There is no empirical evidence to support the sleeper effect. (PsycINFO Database Record
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A reanalysis using longitudinal meta-analytical methods could not replicate the finding that the effects of components were found in the long term.
“A reanalysis using longitudinal meta-analytical methods could not replicate the finding that the effects of components were found in the long term (Flückiger et al. 2015, Wampold & Imel 2015).”
From The Role of Common Factors in Psychotherapy Outcomes (2019) · cites this paper as Flückiger et al. 2015
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · see citing works
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763 · see citing works
Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation
Beck, A. T., Steer, R. A., & Carbin, M. G. (1988) · Clinical Psychology Review · doi:10.1016/0272-7358(88)90050-5 · Cited by 11,696 · see citing works
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,347 · see citing works
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955 · see citing works
Synthesizing standardized mean‐change measures
Becker, B. J. (1988) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1988.tb00901.x · Cited by 889 · see citing works
Ahn, H.-N., & Wampold, B. E. (2001) · Journal of Counseling Psychology · doi:10.1037/0022-0167.48.3.251 · Cited by 571 · see citing works
Self-citationCuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010) · Psychological Medicine · doi:10.1017/s0033291709006114 · Cited by 533 · see citing works
How clients make therapy work: The process of active self-healing.
Bohart, A. C., & Tallman, K. (1999) · American Psychological Association eBooks · doi:10.1037/10323-000 · Cited by 404 · see citing works
Exposure and anxiety management in the treatment of social phobia.
Butler, G., Anne, C., Mary, M., Peter, A., & Michael, G. (1984) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.52.4.642 · Cited by 208 · see citing works
Are the parts as good as the whole? A meta-analysis of component treatment studies.
Bell, E. C., Marcus, D. K., & Goodlad, J. K. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0033004 · Cited by 192 · see citing works
Cognitive therapy with obsessive-compulsive disorder: A comparative evaluation
Emmelkamp, P. M. G., & Beens, H. (1991) · Behaviour Research and Therapy · doi:10.1016/0005-7967(91)90120-r · Cited by 175 · see citing works
Exposure and anxiety management in the treatment of social phobia.
Gillian, B., Anne, C., Mary, M., Peter, A., & Michael, G. (1984) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.52.4.642 · Cited by 157 · see citing works
Relapse prevention program for treatment of obsessive-compulsive disorder.
Hiss, H., Foa, E. B., & Kozak, M. J. (1994) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.62.4.801 · Cited by 146 · see citing works
Guidi, J., Fava, G. A., Fava, M., & Papakostas, G. I. (2011) · Psychological Medicine · doi:10.1017/s0033291710000826 · Cited by 118 · see citing works
Mehta, M. (1990) · British Journal of Psychiatry · doi:10.1192/bjp.157.1.133 · Cited by 104 · see citing works
The effect of attributional processes concerning medication taking on return of fear.
Powers, M. B., Smits, J. A. J., Whitley, D., Bystritsky, A., & Telch, M. J. (2008) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.76.3.478 · Cited by 75 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Role of Common Factors in Psychotherapy Outcomes
Cuijpers, P., Reijnders, M., & Huibers, M. J. H. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050718-095424 · Cited by 748 · see citing works
Cuijpers, P., Cristea, I. A., Karyotaki, E., Reijnders, M., & Hollon, S. D. (2019) · Psychotherapy Research · doi:10.1080/10503307.2017.1395922 · Cited by 140 · see citing works
Wong, Y. J., Owen, J., Gabana, N. T., Brown, J. W., McInnis, S., Toth, P., & Gilman, L. (2018) · Psychotherapy Research · doi:10.1080/10503307.2016.1169332 · Cited by 127 · see citing works
Psychological Interventions for the Fear of Public Speaking: A Meta-Analysis
Ebrahimi, O. V., Pallesen, S., Kenter, R., & Nordgreen, T. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00488 · Cited by 62 · see citing works
Kivlighan, D., Goldberg, S. B., Abbas, M., Pace, B. T., Yulish, N. E., Thomas, J. G., Cullen, M. M., Flückiger, C., & Wampold, B. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.05.003 · Cited by 34 · see citing works
Self-citationFlückiger, C., Del Re, A. C., Munder, T., Heer, S., & Wampold, B. E. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.05.001 · Cited by 34 · see citing works
Self-citationFlückiger, C., & Re, A. C. D. (2016) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796016000780 · Cited by 44 · see citing works
Self-citationTondorf, T., Kaufmann, L., Degel, A., Locher, C., Birkhäuer, J., Gerger, H., Ehlert, U., & Gaab, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0187400 · Cited by 18 · see citing works
Podina, I. R., Vîslă, A., Fodor, L. A., & Flückiger, C. (2019) · Clinical Psychology Review · doi:10.1016/j.cpr.2019.101774 · Cited by 18 · see citing works
Self-citationCella, M., Sedgwick, O., Lawrence, M., Grant, N., Tsapekos, D., Harrison, L., & Wykes, T. (2022) · Psychiatry Research · doi:10.1016/j.psychres.2022.114963 · Cited by 11 · see citing works
Psychotherapy works – An inclusive and affirming view to a modern mental health treatment
Flückiger, C., Willutzki, U., Holtforth, M. G., & Wampold, B. E. (2024) · Clinical Psychology in Europe · doi:10.32872/cpe.11971 · Cited by 12 · see citing works
Self-citationVîslă, A., Allemand, M., & Flückiger, C. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23407 · Cited by 10 · see citing works
Self-citation„Positive Nebenwirkungen“ von Psychotherapie
Hoyer, J. (2016) · Zeitschrift für Klinische Psychologie und Psychotherapie · doi:10.1026/1616-3443/a000370 · Cited by 9 · see citing works
Flückiger, C., Willutzki, U., Holtforth, M. G., & Wampold, B. E. (2024) · Die Psychotherapie · doi:10.1007/s00278-023-00703-4 · Cited by 3 · see citing works
Self-citationNiveau, N., Beaudoin, M., & New, B. (2022) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.2021.40.6.79 · Cited by 4 · see citing works
Discounting Counter-Stereotypical Representations in Entertainment Based on Existing Beliefs
Mustafaj, M., & Cin, S. D. (2024) · Media Psychology · doi:10.1080/15213269.2024.2397024 · Cited by 1 · see citing works
Adding components improves treatment outcomes: Reply to Flückiger et al. (2015).
Marcus, D. K., Bell, E. C., & Mercer, S. H. (2015) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000012 · Cited by 1 · see citing works
Pultz, S., & Goldfinger, E. (2020) · Theory & Psychology · doi:10.1177/0959354320914956 · Cited by 1 · see citing works
Contextual factors in treatment : a meta-analytic and an experimental approach
Birkhäuer, J. (2017) · edoc (University of Basel) · doi:10.5451/unibas-006765849
- Authors
- Flückiger, C., Del Re, A. C., & Wampold, B. E.
- Affiliations
- University of Wisconsin–Madison (US)
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2015-04
- Volume
- 83
- Issue
- 2
- Pages
- 438-442
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W2036246370
- Cited by
- 19
- References
- 17
- Citations by year
- 2024: 3 · 2022: 3 · 2020: 1 · 2019: 2 · 2018: 1 · 2017: 3 · 2016: 3 · 2015: 2 · 2014: 1
- FWCI
- 3.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Consulting and Clinical Psychology (publishedVersion)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion)
- BORIS (University Library Bern) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
- Funders
- Swiss Science National Foundation (PZ00P1_136937)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- sleeper effect, treatment effects over time, publication bias
- MeSH terms
- Humans, Models, Psychological, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
