Which design to evaluate complex interventions? Toward a methodological framework through a systematic review
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Evaluation of complex interventions (CI) is challenging for health researchers and requires innovative approaches. The objective of this work is to present the main methods used to evaluate CI. METHODS: A systematic review of the scientific literature was conducted to identify methods used for the evaluation of CI. We searched MEDLINE via PubMed databases for articles including an evaluation or a pilot study of a complex intervention, published in a ten-year period. Key-words of this research were ("complex intervention*" AND "evaluation"). RESULTS: Among 445 identified articles, 100 research results or protocols were included. Among them, 5 presented 2 different types of design in the same publication, thus our work included 105 designs. Individual randomized controlled trials (IRCT) represented 21.9% (n = 23) of evaluation designs, randomized clinical trials adaptations 44.8% (n = 47), quasi -experimental designs and cohort study 19.0% (n = 20), realist evaluation 6.7% (n = 7) and other cases studies and other approaches 8.6% (n = 9). A process/mechanisms analysis was included in 80% (n = 84) of these designs. CONCLUSION: A range of methods can be used successively or combined at various steps of the evaluation approach. A framework is proposed to situate each of the designs with respect to evaluation questions. The growing interest of researchers in alternative methods and the development of their use must be accompanied by conceptual and methodological research in order to more clearly define their principles of use.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Process evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204 · cited 1×
“In 2015, the MRC provided guidance for process evaluation [26], which constitutes major progress as this guidance addresses the challenge of considering intervention processes and mechanisms as part of the whole evaluation approach.”
MethodFramework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474
Why we need observational studies to evaluate the effectiveness of health care
Black, N. (1996) · BMJ · doi:10.1136/bmj.312.7040.1215 · Cited by 1,598
The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting
Hemming, K., Haines, T. P., Chilton, P. J., Girling, A. J., & Lilford, R. J. (2015) · BMJ · doi:10.1136/bmj.h391 · Cited by 1,410
Process evaluation in randomised controlled trials of complex interventions
Oakley, A., Strange, V., Bonell, C., Allen, E., & Stephenson, J. (2006) · BMJ · doi:10.1136/bmj.332.7538.413 · Cited by 1,247
Explanatory and pragmatic attitudes in therapeutical trials
Schwartz, D., & Lellouch, J. (1967) · Journal of Chronic Diseases · doi:10.1016/0021-9681(67)90041-0 · Cited by 1,185 · cited 1×
“These designs aim to test the effectiveness of interventions in routine clinical practice [35, 36] and therefore maximize their applicability and transferability [30, 35].”
BackgroundEvidence-Based Public Health: Moving Beyond Randomized Trials
Victora, C. G., Habicht, J.-P., & Bryce, J. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.400 · Cited by 1,056
Pfadenhauer, L. M., Gerhardus, A., Mozygemba, K., Lysdahl, K. B., Booth, A., Hofmann, B., Wahlster, P., Polus, S., Burns, J., Brereton, L., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0552-5 · Cited by 1,016
A pragmatic view on pragmatic trials
Patsopoulos, N. A. (2011) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2011.13.2/npatsopoulos · Cited by 844
De Silva, M. J., Breuer, E., Lee, L., Asher, L., Chowdhary, N., Lund, C., & Patel, V. (2014) · Trials · doi:10.1186/1745-6215-15-267 · Cited by 738 · cited 3×
“Last decade has seen the emergence of theory driven RCT that use theory of change (ToC) as a pragmatic framework which describes how the intervention affects change [14].”
BackgroundInfluential
Cited by
Ford-Gilboe, M., Varcoe, C., Scott-Storey, K., Perrin, N., Wuest, J., Wathen, C. N., Case, J., & Glass, N. (2020) · BMC Public Health · doi:10.1186/s12889-020-8152-8 · Cited by 76
Moyle, W., Murfield, J., & Lion, K. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104513 · Cited by 75
Soni, A., Thiyagarajan, A., & Reeve, J. (2023) · Addiction · doi:10.1111/add.15997 · Cited by 41
Leenen, J. P. L., Ardesch, V., Kalkman, C. J., Schoonhoven, L., & Patijn, G. A. (2024) · BJS Open · doi:10.1093/bjsopen/zrad128 · Cited by 36
“In fact, the complex nature of the CMVS intervention and implementation also impedes randomization of two different vital signs monitoring protocols in parallel on the same ward but could also lower the consent rate 62,63 .”
BackgroundPolhuis, K. C. M. M., Vaandrager, L., Soedamah-Muthu, S. S., & Koelen, M. A. (2021) · Health Promotion International · doi:10.1093/heapro/daab020 · Cited by 10
Metadata
- Authors
- Minary, L., Trompette, J., Kivits, J., Cambon, L., Tarquinio, C., & Alla, F.
- Venue
- BMC Medical Research Methodology
- Published
- 2019-05-07
- Volume
- 19
- Issue
- 1
- Pages
- 92-92
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W2947799042
- Cited by
- 137
- References
- 13
- FWCI
- 3.97
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Which design to evaluate complex interventions? Toward a methodological framework through a systematic review. https://app.cochat.ai/papers/10.1186/s12874-019-0736-6
- The paper
- Minary, L., Trompette, J., Kivits, J., Cambon, L., Tarquinio, C., & Alla, F. (2019). Which design to evaluate complex interventions? Toward a methodological framework through a systematic review. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-019-0736-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.