CoChat

Advancing complexity science in healthcare research: the logic of logic models

Open access
Mills, Thomas, Lawton, Rebecca, Sheard, Laura · 2019 · BMC Medical Research Methodology

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Logic models are commonly used in evaluations to represent the causal processes through which interventions produce outcomes, yet significant debate is currently taking place over whether they can describe complex interventions which adapt to context. This paper assesses the logic models used in healthcare research from a complexity perspective. A typology of existing logic models is proposed, as well as a formal methodology for deriving more flexible and dynamic logic models. ANALYSIS: Various logic model types were tested as part of an evaluation of a complex Patient Experience Toolkit (PET) intervention, developed and implemented through action research across six hospital wards/departments in the English NHS. Three dominant types of logic model were identified, each with certain strengths but ultimately unable to accurately capture the dynamics of PET. Hence, a fourth logic model type was developed to express how success hinges on the adaption of PET to its delivery settings. Aspects of the Promoting Action on Research Implementation in Health Services (PARIHS) model were incorporated into a traditional logic model structure to create a dynamic "type 4" logic model that can accommodate complex interventions taking on a different form in different settings. CONCLUSION: Logic models can be used to model complex interventions that adapt to context but more flexible and dynamic models are required. An implication of this is that how logic models are used in healthcare research may have to change. Using logic models to forge consensus among stakeholders and/or provide precise guidance across different settings will be inappropriate in the case of complex interventions that adapt to context. Instead, logic models for complex interventions may be targeted at facilitators to enable them to prospectively assess the settings they will be working in and to develop context-sensitive facilitation strategies. Researchers should be clear as to why they are using a logic model and experiment with different models to ensure they have the correct type.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12874-019-0701-4

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Initial Logic Model for the PET Intervention. The initial logic model focuses on the core levers of the intervention. An early finding of the evaluation was that the PET document was insignificant relative to the facilitation provided by the action researchers. Hence, facilitation skills are included as a key resource and the facilitation activities that supported PET’s delivery are listed in the activities section. The people involved in the study besides the action researchers, including HCPs, patients/PPI reps and PE/corporate teams, are also listed as an intervention resource as they were frequently identified as having contributed to outcomes. In addition, various mechanisms are included in the model that could be seen to be operating through the intervention, notably participation, reflection, feedback and action planning/QI cycles. The factors that moderated PET’s delivery, identified as either constraints on wards that struggled with implementation or enablers for achieving full implementation, are listed in the moderators section. Finally, while the ideal, distal outcome of a fully embedded patient experience system (with sustained improvements to patient experience occurring) was not achieved on any ward, the various proximal outcomes listed in the logic model could be identified across the participating wards
Fig. 1 Initial Logic Model for the PET Intervention. The initial logic model focuses on the core levers of the intervention. An early finding of the evaluation was that the PET document was insignificant relative to the facilitation provided by the action researchers. Hence, facilitation skills are included as a key resource and the facilitation activities that supported PET’s delivery are listed in the activities section. The people involved in the study besides the action researchers, including HCPs, patients/PPI reps and PE/corporate teams, are also listed as an intervention resource as they were frequently identified as having contributed to outcomes. In addition, various mechanisms are included in the model that could be seen to be operating through the intervention, notably participation, reflection, feedback and action planning/QI cycles. The factors that moderated PET’s delivery, identified as either constraints on wards that struggled with implementation or enablers for achieving full implementation, are listed in the moderators section. Finally, while the ideal, distal outcome of a fully embedded patient experience system (with sustained improvements to patient experience occurring) was not achieved on any ward, the various proximal outcomes listed in the logic model could be identified across the participating wardsCC BY · PubMed Central
Fig. 2 A Typology of Logic Models in Healthcare Research. Figure 2 describes logic models as having two key characteristics: firstly, whether they list model factors only or whether they also express the relationships between the factors; and, secondly, whether context is included as part of the model or whether it is omitted. While this typology implies the possibility of four types, most logic models take the form of one of three types. Logic models that model the dynamic interaction between interventions and context, the most appropriate type for complex interventions, are rare, hence the question-mark after type 4
Fig. 2 A Typology of Logic Models in Healthcare Research. Figure 2 describes logic models as having two key characteristics: firstly, whether they list model factors only or whether they also express the relationships between the factors; and, secondly, whether context is included as part of the model or whether it is omitted. While this typology implies the possibility of four types, most logic models take the form of one of three types. Logic models that model the dynamic interaction between interventions and context, the most appropriate type for complex interventions, are rare, hence the question-mark after type 4CC BY · PubMed Central
Fig. 3 Example of a Type 2 logic model. Source: Davidoff et al., 2015 [23]. Some type 2 logic models account for complexity by moving to a higher degree of abstraction, listing intervention mechanisms instead of a precise list of intervention resources and activities (see Fig. 3) [23]. This allows for greater flexibility across settings but the linearity of these models will ensure they still fail to capture how complex interventions are formed by their interaction with context
Fig. 3 Example of a Type 2 logic model. Source: Davidoff et al., 2015 [23]. Some type 2 logic models account for complexity by moving to a higher degree of abstraction, listing intervention mechanisms instead of a precise list of intervention resources and activities (see Fig. 3) [23]. This allows for greater flexibility across settings but the linearity of these models will ensure they still fail to capture how complex interventions are formed by their interaction with contextCC BY · PubMed Central
Fig. 4 Example of a Type 3 Logic Model. This type 3 logic model expresses how a complex intervention works across multiple domains in a single setting, with interlinking actions producing a range of outputs and outcomes (Permission granted for publication by Beverley Slater, Improvement Academy Director)
Fig. 4 Example of a Type 3 Logic Model. This type 3 logic model expresses how a complex intervention works across multiple domains in a single setting, with interlinking actions producing a range of outputs and outcomes (Permission granted for publication by Beverley Slater, Improvement Academy Director)CC BY · PubMed Central
Fig. 5 The PARIHS Framework. Source: Hack et al., 2011 [27]. PARIHS explains the success or failure of implementation projects in terms of an interplay between the evidence used in projects, the receptiveness of the context and whether the appropriate facilitation is provided. This is expressed using a three-way Venn diagram [14, 28–30]
Fig. 5 The PARIHS Framework. Source: Hack et al., 2011 [27]. PARIHS explains the success or failure of implementation projects in terms of an interplay between the evidence used in projects, the receptiveness of the context and whether the appropriate facilitation is provided. This is expressed using a three-way Venn diagram [14, 28–30]CC BY · PubMed Central
Fig. 6 A Type 4 Logic Model for the PET Intervention. This type 4 logic models lists PET’s core intervention mechanisms (rather than a precise list of activities and resources) to allow for variation across settings while the model shape has been designed to address the inadequacies of the initial logic model identified above. 1. Roles – Where the initial model could not convey how the roles of facilitators and ward teams differed in each setting, the two circles of the Venn are designed to convey that roles and relationships must be adapted to fit the willingness and capacity of each ward team to engage. 2. Interaction between the facilitation and moderators – Although the PARIHS model assigns a single Venn circle to context, we have distinguished between contextual moderators that exert an influence from the outer context and the inner context. This is to show the full spectrum of factors that facilitators of PET must respond to for the intervention to succeed, either utilising positive moderators or overcoming negative moderators. 3. Irregular patterns of proximal outcomes – Core proximal and distal outcomes are listed to the centre-right, emerging if the intervention is successfully adapted to context. Context-dependent proximal outcomes linked to efforts to improve the receptiveness of ward settings to the intervention are situated at the top-right and bottom-right, in accordance with whether they target the outer or inner context respectively. The dotted arrows linking the Venn to these contextual, proximal outcomes convey the peripheral nature of these outcomes. 4. Proximal outcomes influencing later success – Finally, the double-headed arrows convey how the emergence of contextual, proximal outcomes can strengthen the work of the project
Fig. 6 A Type 4 Logic Model for the PET Intervention. This type 4 logic models lists PET’s core intervention mechanisms (rather than a precise list of activities and resources) to allow for variation across settings while the model shape has been designed to address the inadequacies of the initial logic model identified above. 1. Roles – Where the initial model could not convey how the roles of facilitators and ward teams differed in each setting, the two circles of the Venn are designed to convey that roles and relationships must be adapted to fit the willingness and capacity of each ward team to engage. 2. Interaction between the facilitation and moderators – Although the PARIHS model assigns a single Venn circle to context, we have distinguished between contextual moderators that exert an influence from the outer context and the inner context. This is to show the full spectrum of factors that facilitators of PET must respond to for the intervention to succeed, either utilising positive moderators or overcoming negative moderators. 3. Irregular patterns of proximal outcomes – Core proximal and distal outcomes are listed to the centre-right, emerging if the intervention is successfully adapted to context. Context-dependent proximal outcomes linked to efforts to improve the receptiveness of ward settings to the intervention are situated at the top-right and bottom-right, in accordance with whether they target the outer or inner context respectively. The dotted arrows linking the Venn to these contextual, proximal outcomes convey the peripheral nature of these outcomes. 4. Proximal outcomes influencing later success – Finally, the double-headed arrows convey how the emergence of contextual, proximal outcomes can strengthen the work of the projectCC BY · PubMed Central
Fig. 7 Strengths and Weaknesses of Logic Models Types. Researchers should be clear as to why they are using a logic model and experiment with different models to ensure they have the correct type
Fig. 7 Strengths and Weaknesses of Logic Models Types. Researchers should be clear as to why they are using a logic model and experiment with different models to ensure they have the correct typeCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 28 references

Research works printed but not linked: 2

  1. Bakewell O, Garbutt A. The use and abuse of the logical framework approach: a review of international development NGO’s experiences. 2015. https://www.alnap.org/help-library/the-use-and-abuse-of-the-logical-framework-approach . Accessed 08 March 2019.
  2. Cochrane. Developing Logic Models, Cochrane Infectious Diseases, Effective Healthcare Research Consortium, 2016

Cited by

See all 190

Metadata

Authors
Mills, T., Lawton, R., & Sheard, L.
Affiliations
Bradford Royal Infirmary (GB)
Corresponding author
Mills, Thomas
Venue
BMC Medical Research Methodology
Published
2019-03-12
Volume
19
Issue
1
Pages
55-55
ISSN
1471-2288 (ISSN-L)
OpenAlex
W2930947912
Cited by
190
References
28
Citations by year
2026: 18 · 2025: 30 · 2024: 32 · 2023: 26 · 2022: 29 · 2021: 32 · 2020: 22 · 2019: 1
FWCI
15.02
Type
journal article
Language
en
Open access
Licence
  • cc-by
  • creativecommons.org/licenses/by/4.0/ (tdm)
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
  • Research Open (London South Bank University) (acceptedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Research Open (London South Bank University) (submittedVersion, cc-by)
Funders
Health Services and Delivery Research Programme (14/156/32)
Topics
  • Social Sciences › Public Administration › Evaluation and Performance Assessment
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords
logic model, complex interventions, complexity science, intervention evaluation, facilitation techniques, health care research
MeSH terms
Data Collection, Delivery of Health Care, Health Resources, Health Services Research, Humans, Logic, Models, Theoretical, Research Design, Qualitative Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Advancing complexity science in healthcare research: the logic of logic models. https://app.cochat.ai/papers/10.1186/s12874-019-0701-4
The paper
Mills, T., Lawton, R., & Sheard, L. (2019). Advancing complexity science in healthcare research: the logic of logic models. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-019-0701-4

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.