The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index; none of the 79 sources its claims cite is checked for support yet.
Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions
Open accessAbstract
The integration of realist evaluation principles within randomised controlled trials (‘realist RCTs’) enables evaluations of complex interventions to answer questions about what works, for whom and under what circumstances. This allows evaluators to better develop and refine mid-level programme theories. However, this is only one phase in the process of developing and evaluating complex interventions. We describe and exemplify how social scientists can integrate realist principles across all phases of the Medical Research Council framework. Intervention development, modelling, and feasibility and pilot studies need to theorise the contextual conditions necessary for intervention mechanisms to be activated. Where interventions are scaled up and translated into routine practice, realist principles also have much to offer in facilitating knowledge about longer-term sustainability, benefits and harms. Integrating a realist approach across all phases of complex intervention science is vital for considering the feasibility and likely effects of interventions for different localities and population subgroups.
Overview
CoChat Summary
TL;DR
This conceptual article argues that realist evaluation principles should be integrated across all phases of the MRC framework for complex interventions, not only in effectiveness trials. It proposes methods for theorising context–mechanism–outcome configurations during development, feasibility/piloting, realist RCTs, and scale-up evaluations.
Key findings
- Realist principles can be applied across intervention development, feasibility and pilot studies, realist RCTs, and scale-up evaluations to ask what works, for whom, and under what circumstances.
- The authors recommend mixed-method evidence synthesis, formative mixed-method multi-case-study research, and pragmatic formative process evaluation to build “3-D” logic models that include contextual dimensions alongside inputs and outcomes.
- Realist RCTs combine randomisation with theory testing, mediation and moderation analyses, and qualitative process data; examples include the Welsh National Exercise Referral Scheme, where effects varied by referral subgroup, and a whole-school restorative approach trial.
- Scale-up evaluation of IMAGE suggested community mobilisation components were often not sustainable in some contexts, refining theory about why effects may have been limited to women themselves. The Public Health Improvement Research Network in Wales supported 122 multidisciplinary research development groups and secured 72 externally funded projects between 2006 and 2014.
Methods
- Design: conceptual and methodological article, not an empirical study; it draws on case examples of realist studies across phases of the MRC framework.
- Data/sample: illustrative examples include a formative study in six further education colleges, a four-arm cluster randomised pilot trial in 12 secondary schools in south Wales, the Welsh NERS trial, a whole-school restorative approach pilot and RCT, and the IMAGE scale-up evaluation.
- Analysis: narrative argument and synthesis of prior literature and case examples to propose how realist principles can be integrated into intervention development, feasibility/piloting, effectiveness trials, and implementation/scale-up studies.
Limitations
- Stated: the major barrier to formally testing CMO configurations within individual studies is that trials often have small sample sizes, powered for primary outcomes but not necessarily for secondary or intermediate process outcomes; trials are rarely designed for mediator or subgroup analyses, and clinical trials units often reject such secondary analyses for fear of false positives and “data dredging.”
- Stated: there is concern that new UK anti-lobbying regulations may limit knowledge exchange between policymakers and researchers.
- Evident from the text: this is a conceptual proposal illustrated by selected examples rather than a systematic review or empirical test, so the recommended approach is not itself evaluated here.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 1 Not valid · 68 Not checked · 28 Own results and interpretations
In the paper
“The original UK Medical Research Council (MRC) framework for evaluating complex interventions recommended sequential phases of development, feasibility testing and evaluation, culminating in the estimation of an effect size via a randomised controlled trial (RCT), prior to wider implementation (Campbell et al., 2000).”
§ Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions > Introduction
What the sources say
- Not checked yetcites Campbell et al., 2000
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The integration of realist evaluation principles within randomised controlled trials (‘realist RCTs’) enables evaluations of complex interventions to answer questions about what works, for whom and under what circumstances.”
- Also stated in: § Introduction · plain
“Progress is being made in integrating complex systems science and realist evaluation principles with RCTs through ‘realist RCT’ designs, to allow evaluators to go beyond simply asking ‘does it work’ and towards more nuanced consideration of what works, for whom and under what circumstances (Bonell et al., 2012).”
§ Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions > Abstract
What the sources say
- Not checked yetcites Bonell et al., 2012
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Introduction · plain
In the paper
“An emphasis purely on aggregate effectiveness also means that we risk developing, evaluating and recommending interventions for implementation that have small population-level benefits at the expense of widening existing inequalities (Whitehead 2007).”
§ Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions > Introduction
What the sources say
- Not checked yetcites Whitehead 2007
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“This allows evaluators to better develop and refine mid-level programme theories.”
§ Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions > Abstract
What the sources say
- Not checked yetcites Bonell et al., 2012
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, there is an inherent compatibility of complex systems science, critical realism and realist evaluation in their mutual commitment to understanding causality within complex environments.”
§ Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions > Introduction
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
44 more claims
Create a free account to see all 97 claims and to verify the 79 references still open.
- Probably not a substantial claim. 27
- It may state two separate claims as one. 23
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some condemned statistically testing for effectiveness [7, 20, 29, 32, 41], as such studies are often underpowered, hence leading to imprecise and potentially misleading estimates of effect sizes [7, 20].”
From Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance (2018) · cites this paper as [29]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For example, some described pilot studies as a smaller version of a future RCT to run in miniature [7, 8, 19, 29, 32–34] and was sometimes associated with a randomised design [32, 34], but not always [7, 8].”
From Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance (2018) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“Sources of guidance endorsed exploratory studies having formative purposes (i.e. refining the intervention and addressing uncertainties related to intervention implementation [13, 15, 29, 31, 39]) as well as summative goals (i.e. assessing the potential impact of an intervention or its promise [6, 13, 39]).”
From Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance (2018) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“Understanding how intervention components interact with various contextual settings [6, 27, 29] and identifying unintended harms [6, 29] as well as potential implementation issues [6, 9, 10, 30] were also highlighted.”
From Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance (2018) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“‘Pilot’ and ‘feasibility’ studies were sometimes used interchangeably [10, 20, 25–28, 31] while others made distinctions between the two according to design features or particular aims [7, 8, 19, 29, 32–34].”
From Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance (2018) · cites this paper as [29]
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Michie, S., van Stralen, M. M., & West, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-42 · Cited by 13,697 · cited 2×
“The dominance of such reductionist methods is associated with the rise of intervention-comparison websites (similar to pricecomparison websites), such as the Blueprints Youth Programmes resource developed in the USA (http://www.blueprintsprograms.com/) and the UK Investing in Children database…”
MethodDeveloping and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995
Social Theory and Social Structure
Ross, E. J., & Merton, R. K. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,315
Social Theory and Social Structure.
Bock, K. E., & Merton, R. K. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,660
Process evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204
Self-citationBhaskar, R. (2013) · Routledge eBooks · doi:10.4324/9780203090732 · Cited by 4,012
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · cited 3×
“…Research Council (MRC) framework for evaluating complex interventions recommended sequential phases of development, feasibility testing and evaluation, culminating in the estimation of an effect size via a randomised controlled trial (RCT), prior to wider implementation (Campbell et al., 2000).”
BackgroundMethodRealist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190 · cited 2×
“‘Realist reviews’ have been suggested as an alternative (or adjunct) to address the lack of focus on CMO configurations in current evidence syntheses (Pawson et al., 2005).”
Background
Research works printed but not linked: 5
- Anderson R, 2008, BMJ, 337, 944
- Griffiths F, 2000, British Journal of General Practice, 50, 913
- Moore G, 2014, Process Evaluation of Complex Interventions: Medical Research Council guidance
- Pawson R, 1997, Realistic Evaluation
- Weiss CH, 1995, New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, 65
Cited by
Noyes, J., Booth, A., Moore, G., Flemming, K., Tunçalp, Ö., & Shakibazadeh, E. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000893 · Cited by 497
Self-citationGuidance for conducting feasibility and pilot studies for implementation trials
Pearson, N., Naylor, P.-J., Ashe, M. C., Fernandez, M., Yoong, S. L., & Wolfenden, L. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00634-w · Cited by 475
O. B. O. T. C.-S. G., Grant, S., Mayo-Wilson, E., Montgomery, P., Macdonald, G., Michie, S., Hopewell, S., & Moher, D. (2018) · Trials · doi:10.1186/s13063-018-2735-z · Cited by 348
Gilmore, B., McAuliffe, E., Power, J., & Vallières, F. (2019) · International Journal of Qualitative Methods · doi:10.1177/1609406919859754 · Cited by 296
Hallingberg, B., Turley, R., Segrott, J., Wight, D., Craig, P., Moore, L., Murphy, S., Robling, M., Simpson, S. A., & Moore, G. (2018) · Pilot and Feasibility Studies · doi:10.1186/s40814-018-0290-8 · Cited by 248
Self-citation
Metadata
- Authors
- Fletcher, A., Jamal, F., Moore, G., Evans, R. E., Murphy, S., & Bonell, C.
- Affiliations
- Cardiff University (GB)
- University College London (GB)
- London School of Hygiene & Tropical Medicine (GB)
- Corresponding author
- Fletcher, Adam
- Venue
- Evaluation
- Publisher
- SAGE Publications
- Published
- 2016-06-02
- Volume
- 22
- Issue
- 3
- Pages
- 286-303
- ISSN
- 1356-3890 (ISSN-L)
- 1461-7153
- PMC
- PMC4946011
- OpenAlex
- W2413220774
- Semantic Scholar
- 2401b5492138e8bbb67ed89aeca50257d4d3f9e1
- Cited by
- 294
- References
- 81
- Citations by year
- 2026: 24 · 2025: 26 · 2024: 24 · 2023: 23 · 2022: 26 · 2021: 64 · 2020: 40 · 2019: 29 · 2018: 24 · 2017: 14
- FWCI
- 24.48
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Evaluation (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ORCA Online Research @Cardiff (Cardiff University) (acceptedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Funders
- Wellcome Trust
- Health and Care Research Wales (SCF-14-09)
- Medical Research Council (MR/K023233/1)
- Topics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- realist evaluation, complex interventions, MRC framework, intervention development, intervention mechanisms, contextual factors, program theory, feasibility study, pilot study, scale-up, sustainability, benefits and harms, routine practice, randomized controlled trials
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions. https://app.cochat.ai/papers/10.1177/1356389016652743
- The paper
- Fletcher, A., Jamal, F., Moore, G., Evans, R. E., Murphy, S., & Bonell, C. (2016). Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions. Evaluation. https://doi.org/10.1177/1356389016652743
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.