Evaluation and validation of social and psychological markers in randomised trials of complex interventions in mental health: a methodological research programme
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Background: The development of the capability and capacity to evaluate the outcomes of trials of complex interventions is a key priority of the National Institute for Health Research (NIHR) and the Medical Research Council (MRC). The evaluation of complex treatment programmes for mental illness (e.g. cognitive–behavioural therapy for depression or psychosis) not only is a vital component of this research in its own right but also provides a well-established model for the evaluation of complex interventions in other clinical areas. In the context of efficacy and mechanism evaluation (EME) there is a particular need for robust methods for making valid causal inference in explanatory analyses of the mechanisms of treatment-induced change in clinical outcomes in randomised clinical trials. Objectives: The key objective was to produce statistical methods to enable trial investigators to make valid causal inferences about the mechanisms of treatment-induced change in these clinical outcomes. The primary objective of this report is to disseminate this methodology, aiming specifically at trial practitioners. Methods: The three components of the research were (1) the extension of instrumental variable (IV) methods to latent growth curve models and growth mixture models for repeated-measures data; (2) the development of designs and regression methods for parallel trials; and (3) the evaluation of the sensitivity/robustness of findings to the assumptions necessary for model identifiability. We illustrate our methods with applications from psychological and psychosocial intervention trials, keeping the technical details to a minimum, leaving the reporting of the more theoretical and mathematically demanding results for publication in appropriate specialist journals. Results: We show how to estimate treatment effects and introduce methods for EME. We explain the use of IV methods and principal stratification to evaluate the role of putative treatment effect mediators and therapeutic process measures. These results are extended to the analysis of longitudinal data structures. We consider the design of EME trials. We focus on designs to create convincing IVs, bearing in mind assumptions needed to attain model identifiability. A key area of application that has become apparent during this work is the potential role of treatment moderators (predictive markers) in the evaluation of treatment effect mechanisms for personalised therapies (stratified medicine). We consider the role of targeted therapies and multiarm trials and the use of parallel trials to help elucidate the evaluation of mediators working in parallel. Conclusions: In order to demonstrate both efficacy and mechanism, it is necessary to (1) demonstrate a treatment effect on the primary (clinical) outcome, (2) demonstrate a treatment effect on the putative mediator (mechanism) and (3) demonstrate a causal effect from the mediator to the outcome. Appropriate regression models should be applied for (3) or alternative IV procedures, which account for unmeasured confounding, provided that a valid instrument can be identified. Stratified medicine may provide a setting where such instruments can be designed into the trial. This work could be extended by considering improved trial designs, sample size considerations and measurement properties. Funding: The project presents independent research funded under the MRC–NIHR Methodology Research Programme (grant reference G0900678).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Davey Smith, G., & Ebrahim, S. (2003) · International Journal of Epidemiology · doi:10.1093/ije/dyg070 · Cited by 5,882
Introduction to Statistical Mediation Analysis
MacKinnon, D. P. (2012) · doi:10.4324/9780203809556 · Cited by 5,618
Mendelian randomization: Using genes as instruments for making causal inferences in epidemiology
Lawlor, D. A., Harbord, R. M., Sterne, J. A. C., Timpson, N., & Davey Smith, G. (2008) · Statistics in Medicine · doi:10.1002/sim.3034 · Cited by 4,618
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230
Avoiding bias from weak instruments in Mendelian randomization studies
Burgess, S., Thompson, S. G., & CRP CHD Genetics Collaboration (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr036 · Cited by 3,952
Mediators and Moderators of Treatment Effects in Randomized Clinical Trials
Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S. (2002) · Archives of General Psychiatry · doi:10.1001/archpsyc.59.10.877 · Cited by 2,562
Cited by
Sleep disturbance and psychiatric disorders
Freeman, D., Sheaves, B., Waite, F., Harvey, A. G., & Harrison, P. J. (2020) · The Lancet Psychiatry · doi:10.1016/s2215-0366(20)30136-x · Cited by 843
Unbiased Scene Graph Generation From Biased Training
Tang, K., Niu, Y., Huang, J., Shi, J., & Zhang, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00377 · Cited by 755
Freeman, D., Sheaves, B., Goodwin, G. M., Yu, L.-M., Nickless, A., Harrison, P. J., Emsley, R., Luik, A. I., Foster, R. G., Wadekar, V., et al. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30328-0 · Cited by 652
Self-citationThe Lancet Psychiatry Commission on psychological treatments research in tomorrow's science
Holmes, E. A., Ghaderi, A., Harmer, C. J., Ramchandani, P. G., Cuijpers, P., Morrison, A. P., Roiser, J. P., Bockting, C. L. H., O'Connor, R. C., Shafran, R., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30513-8 · Cited by 624
Espie, C. A., Emsley, R., Kyle, S. D., Gordon, C., Drake, C. L., Siriwardena, A. N., Cape, J., Ong, J. C., Sheaves, B., Foster, R., et al. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.2745 · Cited by 466
Self-citationSynthesising quantitative evidence in systematic reviews of complex health interventions
Higgins, J. P. T., López-López, J. A., Becker, B. J., Davies, S. R., Dawson, S., Grimshaw, J. M., McGuinness, L. A., Moore, T. H. M., Rehfuess, E. A., Thomas, J., et al. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000858 · Cited by 322
“77 synTHesis ApproACHes for undersTAnding ComplexiTies of THe sysTem Syntheses may seek to address complexities of the system to understand either the impact of the system on the effects of the intervention or the effects of the intervention on the system.”
BackgroundHardy, C., Griffiths, A., Norton, S., & Hunter, M. S. (2018) · Menopause · doi:10.1097/gme.0000000000001048 · Cited by 86
Causal mechanisms in the clinical course and treatment of back pain
Lee, H., Mansell, G., McAuley, J. H., Kamper, S. J., Hübscher, M., Moseley, G. L., Wolfenden, L., Hodder, R. K., & Williams, C. M. (2016) · Best Practice & Research Clinical Rheumatology · doi:10.1016/j.berh.2017.04.001 · Cited by 69
Cashin, A. G., Lee, H., Lamb, S. E., Hopewell, S., Mansell, G., Williams, C. M., Kamper, S. J., Henschke, N., & McAuley, J. H. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.005 · Cited by 36
Sivyer, K., Allen, E., Cooper, Z., Bailey‐Straebler, S., O'Connor, M. E., Fairburn, C. G., & Murphy, R. (2020) · International Journal of Eating Disorders · doi:10.1002/eat.23390 · Cited by 27
Goldsmith, K., Hudson, J. L., Chalder, T., Dennison, L., & Moss-Morris, R. (2020) · Behaviour Research and Therapy · doi:10.1016/j.brat.2020.103594 · Cited by 2
Metadata
- Authors
- Dunn, G., Emsley, R., Liu, H., Landau, S., Green, J., White, I., & Pickles, A.
- Venue
- Health Technology Assessment
- Published
- 2015-11
- Volume
- 19
- Issue
- 93
- Pages
- 1-116
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- OpenAlex
- W2175831798
- Cited by
- 122
- References
- 30
- FWCI
- 8.76
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.