The meaningful assessment of therapy outcomes: Incorporating a qualitative study into a randomized controlled trial evaluating the treatment of adolescent depression.
The DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index.
Abstract
For many years, there have been heated debates about the best way to evaluate the efficacy and effectiveness of psychological therapies. On the one hand, there are those who argue that the randomized controlled trial (RCT) is the only reliable and scientifically credible way to assess psychological interventions. On the other hand, there are those who have argued that psychological therapies cannot be meaningfully assessed using a methodology developed to evaluate the impact of drug treatments, and that the findings of RCTs lack "external validity" and are difficult to translate into routine clinical practice. In this article, we advocate the use of mixed-method research designs for RCTs, combining the rigor of quantitative data about patterns of change with the phenomenological contextualized insights that can be derived from qualitative data. We argue that such an approach is especially important if we wish to understand more fully the impact of therapeutic interventions within complex clinical settings. To illustrate the value of a mixed-method approach, we describe a study currently underway in the United Kingdom, in which a qualitative study (IMPACT-My Experience [IMPACT-ME]) has been "nested" within an RCT (the Improving Mood With Psychoanalytic and Cognitive Behavioral Therapy [IMPACT] study) designed to evaluate the effectiveness of psychological therapies in the treatment of adolescent depression. We argue that such a mixed-methods approach can help us to evaluate the effectiveness of psychological therapies and support the real-world implementation of our findings within increasingly complex and multidisciplinary clinical contexts.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, multiple alternative designs and methods can be useful in shedding light on mechanisms of change, such as single-case experimental studies [80] and qualitative work embedded in RCTs (e.g., [83]).”
From Mediators of outcome in adolescent psychotherapy and their implications for theories and mechanisms of change: a systematic review (2024) · cites this paper as [83]
Citations
Cites
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Mixed Methods Research: A Research Paradigm Whose Time Has Come
Johnson, R. B., & Onwuegbuzie, A. J. (2004) · Educational Researcher · doi:10.3102/0013189x033007014 · Cited by 11,170
Interpretative Phenomenological Analysis: Theory, Method and Research
(2010) · QMiP Bulletin · doi:10.53841/bpsqmip.2010.1.10.44 · Cited by 8,189
SAGE Handbook of Mixed Methods in Social & Behavioral Research
Tashakkori, A., & Teddlie, C. (2010) · doi:10.4135/9781506335193 · Cited by 6,215
Qualitative data analysis for applied policy research
(2002) · Analyzing Qualitative Data · doi:10.4324/9780203413081-14 · Cited by 4,964
Gibbs, G. (2007) · doi:10.4135/9781849208574 · Cited by 4,299
Strategies Of Qualitative Inquiry
Denzin, N. K., & Lincoln, Y. S. (2012) · Cited by 2,900
Mixed methods research designs in counseling psychology.
Hanson, W. E., Creswell, J. W., Clark, V. L. P., Petska, K. S., & Creswell, J. D. (2005) · Journal of Counseling Psychology · doi:10.1037/0022-0167.52.2.224 · Cited by 1,358
The British Child and Adolescent Mental Health Survey 1999: The Prevalence of DSM-IV Disorders
FORD, T., GOODMAN, R., & MELTZER, H. (2003) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200310000-00011 · Cited by 1,227
Foundations of mixed methods research
Tashakkori, A., & Newman, I. (2023) · International Encyclopedia of Education(Fourth Edition) · doi:10.1016/b978-0-12-818630-5.11036-x · Cited by 857
Adult Outcomes of Childhood and Adolescent Depression: I. Psychiatric Status
Harrington, R. (1990) · Archives of General Psychiatry · doi:10.1001/archpsyc.1990.01810170065010 · Cited by 764
Understanding how and why psychotherapy leads to change
Kazdin, A. E. (2009) · Psychotherapy Research · doi:10.1080/10503300802448899 · Cited by 728
Lewin, S., Glenton, C., & Oxman, A. D. (2009) · BMJ · doi:10.1136/bmj.b3496 · Cited by 678
Data Analysis Strategies for Mixed-Method Evaluation Designs
Caracelli, V. J., & Greene, J. C. (1993) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737015002195 · Cited by 627
Evidence-based practices in mental health: Debate and dialogue on the fundamental questions.
(2006) · American Psychological Association eBooks · doi:10.1037/11265-000 · Cited by 567
Adult outcomes of childhood and adolescent depression.
Rc, H. (1991) · Europe PMC (PubMed Central) · Cited by 426
Qualitative Change Process Research on Psychotherapy: Alternative Strategies.
Elliott, R., Slatick, E., & Urman, M. L. (2001) · Cited by 232
Mapping Mixed Methods Research: Methods, Measures, and Meaning
Wheeldon, J. (2010) · Journal of Mixed Methods Research · doi:10.1177/1558689809358755 · Cited by 177
Verhoef, M. J., Casebeer, A. L., & Hilsden, R. J. (2002) · The Journal of Alternative and Complementary Medicine · doi:10.1089/10755530260127961 · Cited by 157
The epidemiology of depression in children and adolescents
Angold, A., & Costello, E. J. (2001) · The Depressed Child and Adolescent · doi:10.1017/cbo9780511543821.007 · Cited by 151
Dattilio, F. M., Edwards, D. J. A., & Fishman, D. B. (2010) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/a0021181 · Cited by 144
Hill, C. E., Chui, H., & Baumann, E. (2013) · Psychotherapy · doi:10.1037/a0030571 · Cited by 114
Lawton, A., & Moghraby, O. S. (2016) · Archives of disease in childhood - Education & practice edition · doi:10.1136/archdischild-2015-308680 · Cited by 102
How to Make Practical Use of Therapeutic Alliance Research in Your Clinical Work
Hilsenroth, M. J., Cromer, T. D., & Ackerman, S. J. (2011) · Humana Press eBooks · doi:10.1007/978-1-60761-792-1_22 · Cited by 96
Blackwood, B., O'Halloran, P., & Porter, S. (2010) · Journal of Research in Nursing · doi:10.1177/1744987110373860 · Cited by 91
Mixed Methods in Health Psychology: Theoretical and Practical Considerations of the Third Paradigm
Dures, E., Rumsey, N., Morris, M., & Gleeson, K. (2011) · Journal of Health Psychology · doi:10.1177/1359105310377537 · Cited by 84
Factors Associated With Early Dropout From Adolescent Psychiatric Outpatient Treatment
PELKONEN, M., MARTTUNEN, M., LAIPPALA, P., & LÖNNQVIST, J. (2000) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200003000-00015 · Cited by 83
Spillane, J. P., Pareja, A. S., Dorner, L. M., Barnes, C. A., May, H., Huff, J., & Camburn, E. M. (2010) · Educational Assessment Evaluation and Accountability · doi:10.1007/s11092-009-9089-8 · Cited by 63
Anger management groups for adolescents: A mixed-methods study of efficacy and treatment preferences
Down, R., Willner, P., Watts, L., & Griffiths, J. (2010) · Clinical Child Psychology and Psychiatry · doi:10.1177/1359104509341448 · Cited by 53
Noyes, J. (2010) · Journal of Research in Nursing · doi:10.1177/1744987110381696 · Cited by 51
Fishman, D. B. (2001) · Applied and Preventive Psychology · doi:10.1016/s0962-1849(01)80004-4 · Cited by 48
Cramer, H., Salisbury, C., Conrad, J., Eldred, J., & Araya, R. (2011) · BMC Psychiatry · doi:10.1186/1471-244x-11-82 · Cited by 45
Rudolph, K. D., & Klein, D. N. (2009) · Development and Psychopathology · doi:10.1017/s0954579409990095 · Cited by 43
Werbart, A., & Levander, S. (2005) · Bulletin of the Menninger Clinic · doi:10.1521/bumc.69.2.103.66507 · Cited by 43
Qualitative research contribution to a randomized clinical trial
Schumacher, K. L., Koresawa, S., West, C. M., Dodd, M. J., Paul, S. M., Tripathy, D., Koo, P., & Miaskowski, C. A. (2005) · Research in Nursing & Health · doi:10.1002/nur.20080 · Cited by 42
Berk, M., Munib, A., Dean, O., Malhi, G. S., Kohlmann, K., Schapkaitz, I., Jeavons, S., Katz, F., Anderson-Hunt, M., Conus, P., et al. (2011) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.09m05741yel · Cited by 33
Philips, B., Wennberg, P., & Werbart, A. (2006) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1348/147608306x128266 · Cited by 31
Putting participation into practice
Guest, D. E., & Knight, K. E. (1979) · Cited by 23
Experiences of overcoming depression in young adults in psychoanalytic psychotherapy
von Below, C., Werbart, A., & Rehnberg, S. (2010) · European Journal of Psychotherapy & Counselling · doi:10.1080/13642537.2010.482745 · Cited by 20
Vuckovic, N. H. (2002) · The Journal of Alternative and Complementary Medicine · doi:10.1089/10755530260127916 · Cited by 19
Qualitative research and the take-up of evidence-based practice
Britten, N. (2010) · Journal of research in nursing · doi:10.1177/1744987110380611 · Cited by 19
A clinical trial of group counselling for changing high-risk sexual behaviour in men
Balmer, D. H., Gikundi, E., Nasio, J., Kihuho, F., & Plummer, F. A. (1998) · Counselling Psychology Quarterly · doi:10.1080/09515079808254041 · Cited by 7
The Role of Qualitative Methods in Outcome Research
McLeod, J. (2001) · SAGE Publications Ltd eBooks · doi:10.4135/9781849209663.n10 · Cited by 7
Hollon, S. D. (2015) · Wiley eBooks · doi:10.1002/9781118625392.wbecp197 · Cited by 3
A multiple case study of psychoanalytic therapies for clients with bulimia nervosa
Lunn, S., Poulsen, S., & Daniel, S. I. F. (2012) · Nordic Psychology · doi:10.1080/19012276.2012.726814 · Cited by 3
Cited by
Watts, R., Day, C., Krzanowski, J., Nutt, D., & Carhart-Harris, R. (2017) · Journal of Humanistic Psychology · doi:10.1177/0022167817709585 · Cited by 589
“Qualitative studies are an important adjunct to quantitative data (Hill, Chui, & Baumann, 2013; Lewin, Glenton, & Oxman, 2009; Midgley, Ansaldo, & Target, 2014).”
Framework analysis: a worked example of a study exploring young people’s experiences of depression
Parkinson, S., Eatough, V., Holmes, J., Stapley, E., & Midgley, N. (2016) · Qualitative Research in Psychology · doi:10.1080/14780887.2015.1119228 · Cited by 390
Self-citationCox, G. R., Callahan, P., Churchill, R., Hunot, V., Merry, S. N., Parker, A. G., & Hetrick, S. E. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008324.pub3 · Cited by 194
O’Keeffe, S., Martin, P., Target, M., & Midgley, N. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00075 · Cited by 156
“In the IMPACTME study, the participants (including adolescents, parents and therapists) from the North London Centre of IMPACT trial were invited to participate in qualitative interviews about their expectations and experiences of therapy (Midgley et al., 2014).”
Self-citationMidgley, N., Mortimer, R., Cirasola, A., Batra, P., & Kennedy, E. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.662671 · Cited by 125
“There is also an increasing need to pay attention to the findings of qualitative research, including studies of client experience and service-user preferences (Midgley et al., 2014).”
BackgroundSelf-citation
Metadata
- Authors
- Midgley, N., Ansaldo, F., & Target, M.
- Affiliations
- University College London (GB)
- ORCID
- Midgley, Nick
- Venue
- Psychotherapy
- Published
- 2014
- Volume
- 51
- Issue
- 1
- Pages
- 128-137
- ISSN
- 0033-3204 (ISSN-L)
- 1939-1536
- OpenAlex
- W2116907639
- Cited by
- 128
- References
- 47
- Citations by year
- 2026: 5 · 2025: 15 · 2024: 14 · 2023: 12 · 2022: 9 · 2021: 13 · 2020: 18 · 2019: 11 · 2018: 7 · 2017: 9 · 2016: 6 · 2015: 7 · 2014: 2
- FWCI
- 5.56
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychotherapy (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- adolescent depression, mixed methods research design, impact studies, psychoanalytic psychotherapy, cognitive behavioral therapy, generalizability, clinical implementation, patient experience
- MeSH terms
- Adolescent, Combined Modality Therapy, Depressive Disorder, Female, United Kingdom, Humans, Long-Term Care, Longitudinal Studies, Male, Psychoanalytic Therapy, Psychotherapy, Brief, Cognitive Behavioral Therapy, Qualitative Research, Secondary Prevention
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.