The DOI matches its Crossref record; 107 of the 107 references it cites were found in a registry or an index.
Evaluation of public health interventions from a complex systems perspective: A research methods review
Open accessAbstract
INTRODUCTION: Applying a complex systems perspective to public health evaluation may increase the relevance and strength of evidence to improve health and reduce health inequalities. In this review of methods, we aimed to: (i) classify and describe different complex systems methods in evaluation applied to public health; and (ii) examine the kinds of evaluative evidence generated by these different methods. METHODS: We adapted critical review methods to identify evaluations of public health interventions that used systems methods. We conducted expert consultation, searched electronic databases (Scopus, MEDLINE, Web of Science), and followed citations of relevant systematic reviews. Evaluations were included if they self-identified as using systems- or complexity-informed methods and if they evaluated existing or hypothetical public health interventions. Case studies were selected to illustrate different types of complex systems evaluation. FINDINGS: Seventy-four unique studies met our inclusion criteria. A framework was developed to map the included studies onto different stages of the evaluation process, which parallels the planning, delivery, assessment, and further delivery phases of the interventions they seek to inform; these stages include: 1) theorising; 2) prediction (simulation); 3) process evaluation; 4) impact evaluation; and 5) further prediction (simulation). Within this framework, we broadly categorised methodological approaches as mapping, modelling, network analysis and 'system framing' (the application of a complex systems perspective to a range of study designs). Studies frequently applied more than one type of systems method. CONCLUSIONS: A range of complex systems methods can be utilised, adapted, or combined to produce different types of evaluative evidence. Further methodological innovation in systems evaluation may generate stronger evidence to improve health and reduce health inequalities in our complex world.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995
Self-citationProcess evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204 · cited 2×
“Process evaluations are a common feature of public health intervention evaluation (Moore et al., 2014), but they do not typically include system maps, modelling or the explicit application of systems theories and concepts.”
BackgroundDixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E., Arthur, A., Harvey, J., Hsu, R., Katbamna, S., Olsen, R., Smith, L., et al. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-35 · Cited by 2,289
Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use
Patton, M. Q. (2010) · Cited by 1,740
The need for a complex systems model of evidence for public health
Rutter, H., Savona, N., Glonti, K., Bibby, J., Cummins, S., Finegood, D. T., Greaves, F., Harper, L., Hawe, P., Moore, L., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)31267-9 · Cited by 1,260
Self-citationAgent-Based and Individual-Based Modeling: A Practical Introduction
Railsback, S. F., & Grimm, V. (2011) · Cited by 1,203
Theorising interventions as events in systems
Hawe, P., Shiell, A., & Riley, T. (2009) · American Journal of Community Psychology · doi:10.1007/s10464-009-9229-9 · Cited by 1,143
Checkland, P. B., & Poulter, J. (2007) · Lancaster EPrints (Lancaster University) · Cited by 731
Complex interventions or complex systems? Implications for health economic evaluation
Shiell, A., Hawe, P., & Gold, L. (2008) · BMJ · doi:10.1136/bmj.39569.510521.ad · Cited by 629
The application of systems thinking in health: why use systems thinking?
Peters, D. H. (2014) · Health Research Policy and Systems · doi:10.1186/1478-4505-12-51 · Cited by 577
Cited by
Evans, R., MacDonald, S., Trubey, R., Noyes, J., Robling, M., Willis, S., Boffey, M., Wooders, C., Vinnicombe, S., & Melendez-Torres, G. J. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02260-y · Cited by 29
Metadata
- Authors
- McGill, E., Er, V., Penney, T., Egan, M., White, M., Meier, P., Whitehead, M., Lock, K., Anderson de Cuevas, R., Smith, R., Savona, N., Rutter, H., Marks, D., de Vocht, F., Cummins, S., Popay, J., & Petticrew, M.
- Venue
- Social Science & Medicine
- Publisher
- Elsevier BV
- Published
- 2021-03
- Volume
- 272
- Pages
- 113697
- ISSN
- 0277-9536 (ISSN-L)
- OpenAlex
- W3119019580
- Cited by
- 257
- References
- 107
- FWCI
- 25.44
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluation of public health interventions from a complex systems perspective: A research methods review. https://app.cochat.ai/papers/10.1016/j.socscimed.2021.113697
- The paper
- McGill, E., Er, V., Penney, T., Egan, M., White, M., Meier, P., Whitehead, M., Lock, K., Anderson de Cuevas, R., Smith, R., Savona, N., Rutter, H., Marks, D., de Vocht, F., Cummins, S., Popay, J., & Petticrew, M. (2021). Evaluation of public health interventions from a complex systems perspective: A research methods review. Social Science & Medicine. https://doi.org/10.1016/j.socscimed.2021.113697
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
