The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Using natural experiments to evaluate population health interventions: new Medical Research Council guidance
Open accessAbstract
Natural experimental studies are often recommended as a way of understanding the health impact of policies and other large scale interventions. Although they have certain advantages over planned experiments, and may be the only option when it is impossible to manipulate exposure to the intervention, natural experimental studies are more susceptible to bias. This paper introduces new guidance from the Medical Research Council to help researchers and users, funders and publishers of research evidence make the best use of natural experimental approaches to evaluating population health interventions. The guidance emphasises that natural experiments can provide convincing evidence of impact even when effects are small or take time to appear. However, a good understanding is needed of the process determining exposure to the intervention, and careful choice and combination of methods, testing of assumptions and transparent reporting is vital. More could be learnt from natural experiments in future as experience of promising but lesser used methods accumulates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995
Self-citationMostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
Self-citationEgger, M., Jüni, P., Bartlett, C., Holenstein, F., & Sterne, J. (2003) · Health Technology Assessment · doi:10.3310/hta7010 · Cited by 1,181
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
For objective causal inference, design trumps analysis
Rubin, D. B. (2008) · The Annals of Applied Statistics · doi:10.1214/08-aoas187 · Cited by 831
The coal gas story. United Kingdom suicide rates, 1960-71.
Kreitman, N. (1976) · Journal of Epidemiology & Community Health · doi:10.1136/jech.30.2.86 · Cited by 451
Smoke-free Legislation and Hospitalizations for Acute Coronary Syndrome
Pell, J. P., Haw, S., Cobbe, S., Newby, D. E., Pell, A. C. H., Fischbacher, C., McConnachie, A., Pringle, S., Murdoch, D., Dunn, F., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa0706740 · Cited by 446
Self-citation
Cited by
Process evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204
Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., et al. (2021) · BMJ · doi:10.1136/bmj.n2061 · Cited by 5,808
Self-citationThe need for a complex systems model of evidence for public health
Rutter, H., Savona, N., Glonti, K., Bibby, J., Cummins, S., Finegood, D. T., Greaves, F., Harper, L., Hawe, P., Moore, L., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)31267-9 · Cited by 1,260
Self-citationNature Contact and Human Health: A Research Agenda
Frumkin, H., Bratman, G. N., Breslow, S. J., Cochran, B., Kahn Jr, P. H., Lawler, J. J., Levin, P. S., Tandon, P. S., Varanasi, U., Wolf, K. L., et al. (2017) · Environmental Health Perspectives · doi:10.1289/ehp1663 · Cited by 1,237
“The more thorny challenge is conceptual: natural experiments are highly susceptible to bias, that is to say, to the tendency for exposure to vary across a study population by factors that are also associated with outcomes (Craig et al. 2012; Rutter 2007).”
BackgroundCraig, P., Katikireddi, S. V., Leyland, A., & Popham, F. (2017) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044327 · Cited by 570
Self-citationWolf, J., Prüss‐Ustün, A., Cumming, O., Bartram, J., Bonjour, S., Cairncross, S., Clasen, T., Colford, J. M., Curtis, V., De France, J., et al. (2014) · Tropical Medicine & International Health · doi:10.1111/tmi.12331 · Cited by 461
Hunter, R. F., Christian, H., Veitch, J., Astell-Burt, T., Hipp, J. A., & Schipperijn, J. (2015) · Social Science & Medicine · doi:10.1016/j.socscimed.2014.11.051 · Cited by 451
Moore, G. F., Evans, R. E., Hawkins, J., Littlecott, H., Melendez-Torres, G. J., Bonell, C., & Murphy, S. (2019) · Evaluation · doi:10.1177/1356389018803219 · Cited by 471
Leatherdale, S. T. (2019) · International Journal of Social Research Methodology · doi:10.1080/13645579.2018.1488449 · Cited by 328
Pratt, M., Sarmiento, O. L., Montes, F., Ogilvie, D., Marcus, B. H., Perez, L. G., & Brownson, R. C. (2012) · The Lancet · doi:10.1016/s0140-6736(12)60736-3 · Cited by 295
Self-citationEffects on mental health of a UK welfare reform, Universal Credit: a longitudinal controlled study
Wickham, S., Bentley, L., Rose, T., Whitehead, M., Taylor-Robinson, D., & Barr, B. (2020) · The Lancet Public Health · doi:10.1016/s2468-2667(20)30026-8 · Cited by 151
Changing Behavior Using the Model of Action Phases
Keller, L., Gollwitzer, P. M., & Sheeran, P. (2020) · The Handbook of Behavior Change · doi:10.1017/9781108677318.006 · Cited by 147
Barlow, P., McKee, M., Basu, S., & Stuckler, D. (2017) · Globalization and Health · doi:10.1186/s12992-017-0240-x · Cited by 140
“This scores studies between 1 and 3 in six components, measuring the strength of: (1) study design, (2) selection bias, (3) confounders, (4) data collection, (5) data analysis, and (6) reporting (see Additional file 1) [32].”
BackgroundPaparini, S., Papoutsi, C., Murdoch, J., Green, J., Petticrew, M., Greenhalgh, T., & Shaw, S. E. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01418-3 · Cited by 131
Self-citationTurley, R., Saith, R., Bhan, N., Rehfuess, E., & Carter, B. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010067.pub2 · Cited by 153
Thomas, E., Wickramasinghe, K., Mendis, S., Roberts, N., & Foster, C. (2015) · BMC Public Health · doi:10.1186/s12889-015-2024-7 · Cited by 109
“Additionally, use of alternative study designs such as natural experimental studies may assist in understanding the population-level impact of large scale stove interventions [39].”
BackgroundHumphreys, D. K., Panter, J., Sahlqvist, S., Goodman, A., & Ogilvie, D. (2016) · Journal of Epidemiology and Community Health · doi:10.1136/jech-2015-206381 · Cited by 84
Self-citationEgan, M., Kearns, A., Katikireddi, S. V., Curl, A., Lawson, K., & Tannahill, C. (2016) · Social Science & Medicine · doi:10.1016/j.socscimed.2016.01.026 · Cited by 79
Self-citationTsiachristas, A., Thomas, T., Leal, J., & Lennox, B. R. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012611 · Cited by 72
Grahn, P., Pálsdóttir, A. M., Ottosson, J., & Jonsdottir, I. H. (2017) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph14111310 · Cited by 60
Applying the ROBINS-I tool to natural experiments: an example from public health
Thomson, H., Craig, P., Hilton-Boon, M., Campbell, M., & Katikireddi, S. V. (2018) · Systematic Reviews · doi:10.1186/s13643-017-0659-4 · Cited by 60
Self-citationImpact of offering cycle training in schools upon cycling behaviour: a natural experimental study
Goodman, A., van Sluijs, E. M. F., & Ogilvie, D. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0356-z · Cited by 46
Self-citationRobinson, M., Geue, C., Lewsey, J., Mackay, D., McCartney, G., Curnock, E., & Beeston, C. (2014) · Addiction · doi:10.1111/add.12701 · Cited by 45
Howden-Chapman, P., Keall, M., Whitwell, K., & Chapman, R. (2020) · Science of The Total Environment · doi:10.1016/j.scitotenv.2019.134408 · Cited by 27
(2019) · doi:10.1002/9781119536604.ch17 · Cited by 26
Self-citationRiley, W. T. (2017) · Health Psychology · doi:10.1037/hea0000464 · Cited by 12
Metadata
- Authors
- Craig, P., Cooper, C., Gunnell, D., Haw, S., Lawson, K., Macintyre, S., Ogilvie, D., Petticrew, M., Reeves, B., Sutton, M., & Thompson, S.
- Venue
- Journal of Epidemiology and Community Health
- Publisher
- BMJ
- Published
- 2012-05-10
- Volume
- 66
- Issue
- 12
- Pages
- 1182-1186
- ISSN
- 0141-7681 (ISSN-L)
- 0143-005X
- 1470-2738
- OpenAlex
- W2110904269
- Cited by
- 912
- References
- 11
- FWCI
- 41.65
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
