Factorial Designs Help to Understand How Psychological Therapy Works
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
A large amount of research time and resources are spent trying to develop or improve psychological therapies. However, treatment development is challenging and time-consuming, and the typical research process followed-a series of standard randomized controlled trials-is inefficient and sub-optimal for answering many important clinical research questions. In other areas of health research, recognition of these challenges has led to the development of sophisticated designs tailored to increase research efficiency and answer more targeted research questions about treatment mechanisms or optimal delivery. However, these innovations have largely not permeated into psychological treatment development research. There is a recognition of the need to understand how treatments work and what their active ingredients might be, and a call for the use of innovative trial designs to support such discovery. One approach to unpack the active ingredients and mechanisms of therapy is the factorial design as exemplified in the Multiphase Optimization Strategy (MOST) approach. The MOST design allows identification of the active components of a complex multi-component intervention (such as CBT) using a sophisticated factorial design, allowing the development of more efficient interventions and elucidating their mechanisms of action. The rationale, design, and potential advantages of this approach will be illustrated with reference to the IMPROVE-2 study, which conducts a fractional factorial design to investigate which elements (e.g., thought challenging, activity scheduling, compassion, relaxation, concreteness, functional analysis) within therapist-supported internet-delivered CBT are most effective at reducing symptoms of depression in 767 adults with major depression. By using this innovative approach, we can first begin to work out what components within the overall treatment package are most efficacious on average allowing us to build an overall more streamlined and potent therapy. This approach also has potential to distinguish the role of specific versus non-specific common treatment components within treatment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Rumination reconsidered: a psychometric analysis.
Treynor, W., Gonzalez, R., & Nolen-Hoeksema, S. (2003) · Cognitive Therapy and Research · doi:10.1023/a:1023910315561 · Cited by 3,808
Construction and factorial validation of a short form of the Self-Compassion Scale
Raes, F., Pommier, E., Neff, K. D., & Van Gucht, D. (2011) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.702 · Cited by 2,997 · cited 1×
“For each treatment component, the putative mediator was related to the primary mechanism which each treatment component is hypothesized to most strongly influence, including rumination (5-item Brooding scale) (64) for the functional analysis component, overgeneralization (adapted Attitudes to Self Scale – Revised) (58…”
BackgroundMediators and Moderators of Treatment Effects in Randomized Clinical Trials
Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S. (2002) · Archives of General Psychiatry · doi:10.1001/archpsyc.59.10.877 · Cited by 2,562 · cited 1×
“(69) and modern causal inference methods.”
A taxonomy of behavior change techniques used in interventions
Abraham, C., & Michie, S. (2008) · Health Psychology · doi:10.1037/0278-6133.27.3.379 · Cited by 2,525 · cited 1×
“Alternatively, the components could relate to lower-level, more discrete elements within the treatment techniques such as the behavioral change techniques outlined in a recent taxonomy (45).”
BackgroundMediators and Mechanisms of Change in Psychotherapy Research
Kazdin, A. E. (2007) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091432 · Cited by 2,504 · cited 3×
“Resolving the active mechanisms and active ingredients of psychological interventions has been repeatedly identified as a major priority for research (4, 7, 10, 11).”
BackgroundConstructive and unconstructive repetitive thought
Watkins, E. R. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.2.163 · Cited by 2,421
Self-citationGilbert, P., & Procter, S. (2006) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.507 · Cited by 1,822
How important are the common factors in psychotherapy? An update
Wampold, B. E. (2015) · World Psychiatry · doi:10.1002/wps.20238 · Cited by 1,728 · cited 1×
“The most important of these include a positive and genuine relationship between the therapist and patient, engendering positive expectancies and hope in the patient, and a convincing rationale that explains the symptoms experienced and gives credible reasons for the treatment to be helpful (15).”
BackgroundLeary, M. R., Tate, E. B., Adams, C. E., Batts Allen, A., & Hancock, J. (2007) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.92.5.887 · Cited by 1,728
The origins and nature of compassion focused therapy
Gilbert, P. (2014) · British Journal of Clinical Psychology · doi:10.1111/bjc.12043 · Cited by 1,593
A component analysis of cognitive-behavioral treatment for depression.
Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E., Koerner, K., Gollan, J. K., Gortner, E., & Prince, S. E. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.2.295 · Cited by 1,413
Rebooting psychotherapy research and practice to reduce the burden of mental illness
Kazdin, A. E., & Blase, S. L. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393527 · Cited by 1,254 · cited 1×
“Because of the high prevalence of common mental health problems, there is also a scalability gap because there are not sufficiently available therapists to tackle the global burden of poor mental health (13).”
BackgroundCollins, L. M., Murphy, S. A., & Strecher, V. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.022 · Cited by 1,082
Behavioral activation treatment for depression: returning to contextual roots
Jacobson, N. S., Martell, C. R., & Dimidjian, S. (2001) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.8.3.255 · Cited by 1,011
A functional analysis of depression.
Ferster, C. B. (1973) · American Psychologist · doi:10.1037/h0035605 · Cited by 918
Flow, performance and moderators of challenge-skill balance
Engeser, S., & Rheinberg, F. (2008) · Motivation and Emotion · doi:10.1007/s11031-008-9102-4 · Cited by 860 · cited 1×
“For each treatment component, the putative mediator was related to the primary mechanism which each treatment component is hypothesized to most strongly influence, including rumination (5-item Brooding scale) (64) for the functional analysis component, overgeneralization (adapted Attitudes to Self Scale – Revised) (58…”
BackgroundA meta-analysis of the effects of cognitive therapy in depressed patients
Gloaguen, V., Cottraux, J., Cucherat, M., & Blackburn, I.-M. (1998) · Journal of Affective Disorders · doi:10.1016/s0165-0327(97)00199-7 · Cited by 809
The Origins and Current Status of Behavioral Activation Treatments for Depression
Dimidjian, S., Barrera, M., Martell, C., Muñoz, R. F., & Lewinsohn, P. M. (2011) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032210-104535 · Cited by 785
The Role of Common Factors in Psychotherapy Outcomes
Cuijpers, P., Reijnders, M., & Huibers, M. J. H. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050718-095424 · Cited by 748 · cited 3×
“As such, they meet the requirement for delineating active components raised by multiple commentators (8, 10, 14).”
BackgroundThe Lancet Psychiatry Commission on psychological treatments research in tomorrow's science
Holmes, E. A., Ghaderi, A., Harmer, C. J., Ramchandani, P. G., Cuijpers, P., Morrison, A. P., Roiser, J. P., Bockting, C. L. H., O'Connor, R. C., Shafran, R., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30513-8 · Cited by 624
Cuijpers, P., Karyotaki, E., Weitz, E., Andersson, G., Hollon, S. D., & van Straten, A. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.02.026 · Cited by 586
Andrews, G., Issakidis, C., Sanderson, K., Corry, J., & Lapsley, H. (2004) · British Journal of Psychiatry · doi:10.1192/bjp.184.6.526 · Cited by 560 · cited 1×
“Further, it is estimated that current treatments, if delivered optimally, would only reduce the burden of depression by one third (6).”
BackgroundPsychological treatment of generalized anxiety disorder: A meta-analysis
Cuijpers, P., Sijbrandij, M., Koole, S., Huibers, M., Berking, M., & Andersson, G. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.01.002 · Cited by 527
Collins, L. M., Dziak, J. J., & Li, R. (2009) · Psychological Methods · doi:10.1037/a0015826 · Cited by 431 · cited 1×
“In interpreting the results, the assumption is that the 3-way and higher interactions are highly likely to be negligible, based on extensive research and principles within factorial experiment research (27, 63).”
BackgroundWatkins, E. R., Mullan, E., Wingrove, J., Rimes, K., Steiner, H., Bathurst, N., Eastman, R., & Scott, J. (2011) · British Journal of Psychiatry · doi:10.1192/bjp.bp.110.090282 · Cited by 410
Self-citationCollins, L. M., Dziak, J. J., Kugler, K. C., & Trail, J. B. (2014) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2014.06.021 · Cited by 335 · cited 2×
“Advantage 4: Factorial Designs Are Efficient and Economical Factorial designs are efficient and economical compared to alternative designs such as individual experiments and single factor designs because they often require substantially fewer trials and participants to achieve the same statistical power for component…”
BackgroundA Conceptual and Methodological Analysis of the Nonspecifics Argument.
DeRubeis, R. J., Brotman, M. A., & Gibbons, C. J. (2005) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.bpi022 · Cited by 310
A meta-(re) analysis of the effects of cognitive therapy versus ‘other therapies’ for depression
Wampold, B. E., Minami, T., Baskin, T. W., & Callen Tierney, S. (2002) · Journal of Affective Disorders · doi:10.1016/s0165-0327(00)00287-1 · Cited by 305
The Multiphase Optimization Strategy for Engineering Effective Tobacco Use Interventions
Collins, L. M., Baker, T. B., Mermelstein, R. J., Piper, M. E., Jorenby, D. E., Smith, S. S., Christiansen, B. A., Schlam, T. R., Cook, J. W., & Fiore, M. C. (2011) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9253-x · Cited by 300
The Behavioral Activation for Depression Scale–Short Form: Development and Validation
Manos, R. C., Kanter, J. W., & Luo, W. (2011) · Behavior Therapy · doi:10.1016/j.beth.2011.04.004 · Cited by 276
Web-Based Smoking-Cessation Programs
Strecher, V. J., McClure, J. B., Alexander, G. L., Chakraborty, B., Nair, V. N., Konkel, J. M., Greene, S. M., Collins, L. M., Carlier, C. C., Wiese, C. J., et al. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.12.024 · Cited by 266 · cited 2×
“MOST is well-validated (29, 30, 34, 40) and recommended within the Medical Research Council Complex Intervention guidelines (41, 42).”
BackgroundMethodRoth, A. D., & Pilling, S. (2008) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465808004141 · Cited by 260 · cited 1×
“Three of the components chosen had been identified as elements for CBT for depression, using a Delphi technique (46): applied relaxation; activity monitoring and scheduling; detecting and reality testing automatic thoughts.”
MethodPsychological treatments: A call for mental-health science
Holmes, E. A., Craske, M. G., & Graybiel, A. M. (2014) · Nature · doi:10.1038/511287a · Cited by 214
Common versus specific factors in psychotherapy: opening the black box
Mulder, R., Murray, G., & Rucklidge, J. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30100-1 · Cited by 198
Are the parts as good as the whole? A meta-analysis of component treatment studies.
Bell, E. C., Marcus, D. K., & Goodlad, J. K. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0033004 · Cited by 192 · cited 1×
“Component Study Designs One experimental approach that has been used to examine the specific elements of psychological interventions is the component study (9), in which the full intervention is compared with the intervention with at least one component removed (a dismantling study) or in which a component is added to…”
MethodWatkins, E. R., Taylor, R. S., Byng, R., Baeyens, C., Read, R., Pearson, K., & Watson, L. (2012) · Psychological Medicine · doi:10.1017/s0033291711002480 · Cited by 160
Self-citationIs the Dodo bird endangered in the 21st century? A meta-analysis of treatment comparison studies
Marcus, D. K., O'Connell, D., Norris, A. L., & Sawaqdeh, A. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.08.001 · Cited by 148 · cited 1×
“Some comparative RCTs and meta-analyses have found that one therapy has outperformed another therapy (17, 18), which proponents of specific factors have argued as evidence for specific treatment effects.”
BackgroundDziak, J. J., Nahum-Shani, I., & Collins, L. M. (2012) · Psychological Methods · doi:10.1037/a0026972 · Cited by 142
Cuijpers, P., Cristea, I. A., Karyotaki, E., Reijnders, M., & Hollon, S. D. (2019) · Psychotherapy Research · doi:10.1080/10503307.2017.1395922 · Cited by 140
Developing multicomponent interventions using fractional factorial designs
Chakraborty, B., Collins, L. M., Strecher, V. J., & Murphy, S. A. (2009) · Statistics in Medicine · doi:10.1002/sim.3643 · Cited by 107
Collins, L. M., Chakraborty, B., Murphy, S. A., & Strecher, V. (2009) · Clinical Trials · doi:10.1177/1740774508100973 · Cited by 80
Research works printed but not linked: 26
- Cuijpers, 2014, Comparison of psychotherapies for adult depression to pill placebo control groups: a meta-analysis, psychol Med, 44, 10.1017/S0033291713000457 · DOI · Paper page
- Hollon, 2002, Psychosocial intervention development for the prevention and treatment of depression: Promoting innovation and increasing access, Biol Psychiatry, 52, 10.1016/S0006-3223(02)01384-7 · DOI · Paper page
- 2015, Psychosocial interventions for mental and substance use disorders: A framework for establishing evidence-based standards.
- Kazdin, 2014, Evidence-based psychotherapies I: qualifiers and limitations in what we know, South Afr J Psychol, 44, 381, 10.1177/0081246314533750 · DOI · Paper page
- Baskin, 2003, Establishing specificity in psychotherapy: A meta-analysis of structural equivalence of placebo controls, J Consulting Clin Psychol, 71, 10.1037/0022-006X.71.6.973 · DOI · Paper page
- Baker, 2017, Implementing Clinical Research Using Factorial Designs: A Primer, Behav Ther, 48, 10.1016/j.beth.2016.12.005 · DOI · Paper page
- Gwadz, 2017, Using the multiphase optimization strategy (MOST) to optimize an HIV care continuum intervention for vulnerable populations: a study protocol, BMC Public Health, 17, 383, 10.1186/s12889-017-4279-7 · DOI · Paper page
- Piper, 2016, Identifying effective intervention components for smoking cessation: a factorial screening experiment, Addiction, 111, 10.1111/add.13162 · DOI · Paper page
- Uwatoko, 2018, Healthy Campus Trial: a multiphase optimization strategy (MOST) fully factorial trial to optimize the smartphone cognitive behavioral therapy (CBT) app for mental health promotion among university students: study protocol for a randomized controlled trial, Trials, 19, 353, 10.1186/s13063-018-2719-z · DOI · Paper page
- Watkins, 2016, Implementing multifactorial psychotherapy research in online virtual environments (IMPROVE-2): study protocol for a phase III trial of the MOST randomized component selection method for internet cognitive-behavioural therapy for depression, BMC Psychiatry, 16, 13, 10.1186/s12888-016-1054-8 · DOI · Paper page
- Wilbur, 2016, Utilizing MOST frameworks and SMART designs for intervention research, Nurs Outlook, 64, 10.1016/j.outlook.2016.04.005 · DOI
- Collins, 2005, A strategy for optimizing and evaluating behavioral interventions, Ann Behav Med, 30, 65, 10.1207/s15324796abm3001_8 · DOI · Paper page
- Spring, 2018, Optimization Experiments In The Field: The Most Framework Through 3 Clinical Trials, Ann Behav Med, 52, 10.1093/abm/kay013 · DOI
- Collins, 2016, Optimization of Multicomponent Behavioral and Biobehavioral Interventions for the Prevention and Treatment of HIV/AIDS, AIDS Behav, 20, 10.1007/s10461-015-1145-4 · DOI · Paper page
- Piper, 2018, A Randomized Controlled Trial of an Optimized Smoking Treatment Delivered in Primary Care, Ann Behav Med, 52, 10.1093/abm/kax059 · DOI
- Schlam, 2016, Comparative effectiveness of intervention components for producing long-term abstinence from smoking: a factorial screening experiment, Addiction, 111, 10.1111/add.13153 · DOI · Paper page
- Craig, 2008, Developing and evaluating complex interventions: the new Medical Research Council guidance, Br Med J, 337, 10.1136/bmj.a1655 · DOI · Paper page
- Watkins, 2014, A Habit-Goal Framework of Depressive Rumination, J Abnormal Psychol, 123, 24, 10.1037/a0035540 · DOI · Paper page
- Beck, 1976, Cognitive Therapy and the Emotional Disorders.
- Beck, 1967, Depression: Clinical, Experimental and Theoretical Aspects.
- Beck ATR, 1979, Cognitive Therapy of Depression.
- Watkins, 2008, Processing mode causally influences emotional reactivity: distinct effects of abstract versus concrete construal on emotional response, Emotion., 8, 10.1037/1528-3542.8.3.364 · DOI · Paper page
- Carver, 1998, Generalization, adverse events, and development of depressive symptoms, J Pers, 66, 10.1111/1467-6494.00026 · DOI · Paper page
- Matos, 2017, Psychological and Physiological Effects of Compassionate Mind Training: a Pilot Randomised Controlled Study, Mindfulness, 8, 10.1007/s12671-017-0745-7 · DOI · Paper page
- Wu, 2011, Experiments: planning, analysis, and optimization.
- Hollon, 1980, Cognitive Self-Statements In Depression - Development Of An Automatic Thoughts Questionnaire, Cogn Ther Res, 4, 10.1007/BF01178214 · DOI · Paper page
Cited by
Spanhel, K., Balci, S., Feldhahn, F., Bengel, J., Baumeister, H., & Sander, L. B. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00498-1 · Cited by 178
Measuring the active elements of cognitive-behavioral therapies
Cohen, Z. D., Barnes-Horowitz, N. M., Forbes, C. N., & Craske, M. G. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104364 · Cited by 58
Andersson, G., Käll, A., Juhlin, S., Wahlström, C., de Fine Licht, E., Färdeman, S., Franck, A., Tholcke, A., Nachtweij, K., Fransson, E., et al. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104265 · Cited by 51
Watkins, E., Newbold, A., Tester-Jones, M., Collins, L. M., & Mostazir, M. (2023) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2023.1937 · Cited by 40
Self-citationSpanhel, K., Hovestadt, E., Lehr, D., Spiegelhalder, K., Baumeister, H., Bengel, J., & Sander, L. B. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.832196 · Cited by 37
“the intervention in a sufficiently powered randomized controlled trial (63, 67, 99), or in randomized factorial trials (100, 101).”
Background
Metadata
- Authors
- Watkins, E. R., & Newbold, A.
- Venue
- Frontiers in Psychiatry
- Publisher
- Frontiers Media SA
- Published
- 2020-05-14
- Volume
- 11
- Pages
- 429-429
- ISSN
- 1664-0640 (ISSN-L)
- OpenAlex
- W3024950821
- Cited by
- 51
- References
- 42
- FWCI
- 5.40
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.