Do health improvement programmes fit with MRC guidance on evaluating complex interventions?
Abstract
Although planning of new health policy could be improved to enable more robust evaluation, Mhairi Mackenzie and colleagues argue that randomised controlled trials are not always suitable or practical
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Public health interventions may not reach the level of stability required to conduct evaluations such as randomised controlled trials.
“Therefore, they may not reach the level of stability required to conduct evaluations such as randomised controlled trials (RCTs) [1].”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [1]
Cites
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · see citing works
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · see citing works
Developing and Evaluating Complex Interventions: New Guidance
Craig, P., Dieppe, P. A., MacIntyre, S., Michie, S., Nazareth, I., & Petticrew, M. P. (2006) · UCL Discovery (University College London) · Cited by 1,877 · see citing works
Complex interventions: how “out of control” can a randomised controlled trial be?
Hawe, P., Shiell, A., & Riley, T. (2004) · BMJ · doi:10.1136/bmj.328.7455.1561 · Cited by 1,244 · see citing works
Theorising interventions as events in systems
Hawe, P., Shiell, A., & Riley, T. (2009) · American Journal of Community Psychology · doi:10.1007/s10464-009-9229-9 · Cited by 1,143 · see citing works
Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions
Rogers, P. J. (2008) · Evaluation · doi:10.1177/1356389007084674 · Cited by 999 · see citing works
Evaluation : methods for studying programs and policies
Weiss, C. H. (1997) · Cited by 931 · see citing works
O'Neil, J. H. (1995) · North East Linguistics Society · Cited by 785 · see citing works
May, C., Finch, T., Mair, F., Ballini, L., Dowrick, C., Eccles, M., Gask, L., MacFarlane, A., Murray, E., Rapley, T., et al. (2007) · BMC Health Services Research · doi:10.1186/1472-6963-7-148 · Cited by 768 · see citing works
Complex interventions or complex systems? Implications for health economic evaluation
Shiell, A., Hawe, P., & Gold, L. H. (2008) · BMJ · doi:10.1136/bmj.39569.510521.ad · Cited by 630 · see citing works
Quality of Primary Care in England with the Introduction of Pay for Performance
Campbell, S., Reeves, D., Kontopantelis, E., Middleton, E., Sibbald, B., & Roland, M. O. (2007) · New England Journal of Medicine · doi:10.1056/nejmsr065990 · Cited by 426 · see citing works
Evidence for public health policy on inequalities: 1: The reality according to policymakers
Petticrew, M. P., Whitehead, M., Macintyre, S. J., Graham, H., & Egan, M. (2004) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2003.015289 · Cited by 380 · see citing works
Experimentation and social interventions: a forgotten but important history
Oakley, A. (1998) · BMJ · doi:10.1136/bmj.317.7167.1239 · Cited by 150 · see citing works
Twenty five years of case finding and audit in a socially deprived community.
Hart, J. T., Thomas, C., Gibbons, B., Edwards, C. C., Hart, M. M., Jones, J., Jones, M., & Walton, P. (1991) · BMJ · doi:10.1136/bmj.302.6791.1509 · Cited by 90 · see citing works
Evaluating service delivery interventions to enhance patient safety
Brown, C., & Lilford, R. J. (2008) · BMJ · doi:10.1136/bmj.a2764 · Cited by 62 · see citing works
Sowden, S., & Raine, R. (2008) · Journal of Epidemiology and Community Health · doi:10.1136/jech.2007.069781 · Cited by 57 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Using theory of change to design and evaluate public health interventions: a systematic review
Breuer, E., Lee, L., De Silva, M., & Lund, C. (2015) · Implementation Science · doi:10.1186/s13012-016-0422-6 · Cited by 425 · see citing works
Wells, M., Williams, B., Treweek, S., Coyle, J., & Taylor, J. (2012) · Trials · doi:10.1186/1745-6215-13-95 · Cited by 297 · see citing works
- Authors
- Mackenzie, M., O'Donnell, C., Halliday, E., Sridharan, S., & Platt, S.
- Affiliations
- University of Glasgow (GB)
- National Health Service Scotland (GB)
- St. Michael's Hospital (CA)
- University of Edinburgh (GB)
- Corresponding author
- Mackenzie, M.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2010-02-01
- Volume
- 340
- Issue
- feb01 1
- Pages
- c185-c185
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- OpenAlex
- W2046096770
- Cited by
- 127
- References
- 16
- Citations by year
- 2025: 1 · 2023: 4 · 2022: 1 · 2021: 3 · 2020: 3 · 2019: 4 · 2017: 4 · 2016: 4 · 2015: 7 · 2014: 3 · 2013: 71 · 2012: 8
- FWCI
- 40.40
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Lancaster EPrints (Lancaster University) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- complex interventions, health improvement programmes, randomized controlled trials, health policy planning
- MeSH terms
- Delivery of Health Care, Health Policy, Health Promotion, Humans, Program Evaluation, Program Development, Practice Guidelines as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
