Guidance on how to develop complex interventions to improve health and healthcare
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index; 1 of the 10 sources its claims cite were checked for support.
Abstract
Objective: To provide researchers with guidance on actions to take during intervention development. Summary of key points: Based on a consensus exercise informed by reviews and qualitative interviews, we present key principles and actions for consideration when developing interventions to improve health. These include seeing intervention development as a dynamic iterative process, involving stakeholders, reviewing published research evidence, drawing on existing theories, articulating programme theory, undertaking primary data collection, understanding context, paying attention to future implementation in the real world and designing and refining an intervention using iterative cycles of development with stakeholder input throughout. Conclusion: Researchers should consider each action by addressing its relevance to a specific intervention in a specific context, both at the start and throughout the development process.
Overview
CoChat Summary
TL;DR
This paper offers consensus-based guidance on how to develop complex health interventions, framing development as a dynamic, iterative process rather than a fixed sequence. It argues that developers should continually judge which actions matter for a specific intervention and context, from the start through later refinement and implementation.
Key findings
- The guidance is based on a consensus exercise informed by reviews and qualitative interviews, and it recommends considering intervention development as “dynamic,” “iterative,” “creative,” “open to change,” and “forward looking.”
- The paper identifies a set of key actions for development, including involving stakeholders throughout, reviewing published research evidence, drawing on existing theories, articulating programme theory, undertaking primary data collection, understanding context, and planning for future implementation.
- The authors state there is “insufficient research evidence” to recommend that any particular published approach or set of actions is essential to produce a successful intervention.
- The paper emphasizes that iterative refinement should continue until later iterations produce “few changes,” using quantitative and qualitative methods to assess acceptability, feasibility, desirability, and potential unintended harms.
Methods
- The paper is a guidance article built from triangulation of three sources: reviews of published approaches to intervention development and primary research on intervention development, qualitative interviews, and consensus exercises.
- The consensus process included two simultaneous and identical e-Delphi studies for intervention developers and wider stakeholders, followed by a consensus workshop; participants rated 85 items on a five-point scale.
- An international expert panel with members from the UK, USA, and Europe guided the research and took part in the e-Delphi studies and workshop.
Limitations
- The paper explicitly states that the evidence base is sparse and that there is insufficient research evidence to identify a single best published approach or essential set of actions.
- It notes that some aspects of the guidance may not be relevant to some interventions or contexts, and that not all developers have large resources, so the guidance must be used flexibly.
- Evident from the text: the guidance is largely based on expert opinion and consensus rather than strong direct evidence that specific development actions improve intervention success.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 0 Not valid · 9 Not checked · 27 Own results and interpretations
In the paper
“a review of reviews shows that interventions developed with existing theory do not result in more effective interventions than those not using existing theory.10”
§ Guidance on how to develop complex interventions to improve health and healthcare > Framework for intervention development
What the sources say
- Not checked yetcites [10] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The rationale for understanding this context, and developing interventions which can operate within it, is to avoid developing interventions that fail during later evaluation because too few people deliver or use them.”
§ Guidance on how to develop complex interventions to improve health and healthcare > Framework for intervention development > Key actions of intervention development
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is insufficient research evidence to recommend that a particular published approach or set of actions is essential to produce a successful intervention.”
§ Guidance on how to develop complex interventions to improve health and healthcare > Conclusions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
In the paper
“We base this guidance on expert opinion because there is a research evidence gap about which actions are needed in intervention development to produce successful health interventions.”
- Also stated in: § Framework for intervention development · plain
“Unfortunately, this evidence base is sparse in the field of health, largely due to the difficulty of empirically addressing this question.8 9”
§ Guidance on how to develop complex interventions to improve health and healthcare > Framework for intervention development
What the sources say
- Not checked yetcites 8 9
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Framework for intervention development · plain
In the paper
“Many of these approaches share the same actions4 6 and simply place more emphasis on one or a subset of actions.”
§ Guidance on how to develop complex interventions to improve health and healthcare > Framework for intervention development > Key actions of intervention development
What the sources say
- Not checked yetcites 4 6
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
9 references still to check
A free account verifies the references that are still open.
- Probably not a substantial claim. 30
- It may state two separate claims as one. 12
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent guidelines suggest that complex health interventions should be feasible, acceptable, engaging, transferable to other settings, and relevant [37].”
From Feasibility and Preliminary Efficacy of Web-Based and Mobile Interventions for Common Mental Health Problems in Working Adults: Multi-Arm Randomized Pilot Trial (2022) · cites this paper as [37]
Experimental — accuracy not yet measuredIn the citing paper
“Intervention developers create effective interventions, then seek to convince funders to pay for them [59].”
From Translating economic evaluations into financing strategies for implementing evidence-based practices (2021) · cites this paper as [59]
In the citing paper
“complex intervention research does not always begin with new or researcher led interventions.”
From A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance (2021) · cites this paper as [4]
Citations
Cites
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · cited 2×
“The UK Medical Research Council (MRC) published influential guidance on developing and evaluating complex interventions, presenting a framework of four phases: development, feasibility/piloting, evaluation and implementation.(1) The development phase is what happens between the idea for an intervention and formal pilo…”
BackgroundMethodHoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548
Process evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204 · cited 1×
“Indeed, they are advocated for use in process evaluations alongside outcome evaluations in the recent MRC Guidance on process evaluation.(15)”
MethodSelf-citationPlanning health promotion programs an intervention mapping approach
Bartholomew, L. K., Parcel, G. S., Kok, G., & Gottlieb, N. (2006) · Research Publications (Maastricht University) · Cited by 2,291
Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions
Rogers, P. J. (2008) · Evaluation · doi:10.1177/1356389007084674 · Cited by 999 · cited 1×
“A programme theory describes how a specific intervention is expected to lead to its effects and under what conditions.(14) It shows the causal pathways between the content of the intervention, intermediate outcomes and long-term goals and how these interact with contextual factors.”
BackgroundDemystifying theory and its use in improvement
Davidoff, F., Dixon-Woods, M., Leviton, L., & Michie, S. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003627 · Cited by 714
A Description of Think Aloud Method and Protocol Analysis
Fonteyn, M. E., Kuipers, B., & Grobe, S. J. (1993) · Qualitative Health Research · doi:10.1177/104973239300300403 · Cited by 670
Taxonomy of approaches to developing interventions to improve health: a systematic methods overview
O’Cathain, A., Croot, L., Sworn, K., Duncan, E., Rousseau, N., Turner, K., Yardley, L., & Hoddinott, P. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0425-6 · Cited by 576
Self-citationMoore, G. F., Evans, R. E., Hawkins, J., Littlecott, H., Melendez-Torres, G. J., Bonell, C., & Murphy, S. (2019) · Evaluation · doi:10.1177/1356389018803219 · Cited by 471 · cited 1×
“Context also includes the wider complex health and social care, societal or political systems within which any intervention will operate.(18) Different approaches can be taken to understand context, including reviews of evidence, stakeholder engagement and primary data collection.”
BackgroundCraig, P., Di Ruggiero, E., Frohlich, K. L., Mykhalovskiy, E., White, M., on behalf of the Canadian Institutes of Health Research (CIHR)–National Institute for Health Research (NIHR) Context Guidance Authors Group (listed alphabetically), Campbell, R., Cummins, S., Edwards, N., Hunt, K., et al. (2018) · doi:10.3310/cihr-nihr-01 · Cited by 370
Cited by
Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., et al. (2021) · BMJ · doi:10.1136/bmj.n2061 · Cited by 5,808
Sheldon, E., Simmonds-Buckley, M., Bone, C., Mascarenhas, T., Chan, N., Wincott, M., Gleeson, H., Sow, K., Hind, D., & Barkham, M. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.03.054 · Cited by 644
Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., et al. (2021) · Health Technology Assessment · doi:10.3310/hta25570 · Cited by 522
Adapting interventions to new contexts—The ADAPT guidance
Moore, G., Campbell, M., Copeland, L., Craig, P., Movsisyan, A., Hoddinott, P., Littlecott, H., O’Cathain, A., Pfadenhauer, L., Rehfuess, E., et al. (2021) · BMJ · doi:10.1136/bmj.n1679 · Cited by 511
Self-citationDuncan, E., O'Cathain, A., Rousseau, N., Croot, L., Sworn, K., Turner, K. M., Yardley, L., & Hoddinott, P. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-033516 · Cited by 481
Self-citation
Metadata
- Authors
- O"Cathain, A., Croot, L., Duncan, E., Rousseau, N., Sworn, K., Turner, K. M., Yardley, L., & Hoddinott, P.
- Affiliations
- Medical Care Research Unit, School of Health and Related Research, University of Sheffield, Sheffield, UK
- University of Sheffield (GB)
- University of Stirling (GB)
- University of Bristol (GB)
- ORCID
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2019-08
- Volume
- 9
- Issue
- 8
- Pages
- e029954
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC6701588
- OpenAlex
- W2958481062
- Cited by
- 1,591
- References
- 18
- Citations by year
- 2026: 150 · 2025: 264 · 2024: 347 · 2023: 249 · 2022: 247 · 2021: 214 · 2020: 114 · 2019: 6
- FWCI
- 197.91
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMJ Open (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- ePrints Soton (University of Southampton) (acceptedVersion)
- Bristol Research (University of Bristol) (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Stirling Online Research Repository (University of Stirling) (submittedVersion, cc-by)
- HighWire Press Open Archive (submittedVersion)
- Funders
- Medical Research Council (MR/N015339/1)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- complex interventions, intervention development, iterative development, stakeholder engagement, program theory, context, theory-based design, primary data collection, health service improvement
- MeSH terms
- Stakeholder Participation, Delivery of Health Care, Health Promotion, Humans, Research Design, Research Personnel, Consensus
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Guidance on how to develop complex interventions to improve health and healthcare. https://app.cochat.ai/papers/10.1136/bmjopen-2019-029954
- The paper
- O"Cathain, A., Croot, L., Duncan, E., Rousseau, N., Sworn, K., Turner, K. M., Yardley, L., & Hoddinott, P. (2019). Guidance on how to develop complex interventions to improve health and healthcare. BMJ Open. https://doi.org/10.1136/bmjopen-2019-029954
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
