Cited by 1,016Health Policy Implementation ScienceEvaluation and Performance AssessmentMeta-analysis and systematic reviews

The DOI matches its Crossref record; 96 of the 99 references it cites were found in a registry or an index; 4 of the 46 sources its claims cite were checked for support.

Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework

Open access

Abstract

BACKGROUND: The effectiveness of complex interventions, as well as their success in reaching relevant populations, is critically influenced by their implementation in a given context. Current conceptual frameworks often fail to address context and implementation in an integrated way and, where addressed, they tend to focus on organisational context and are mostly concerned with specific health fields. Our objective was to develop a framework to facilitate the structured and comprehensive conceptualisation and assessment of context and implementation of complex interventions. METHODS: The Context and Implementation of Complex Interventions (CICI) framework was developed in an iterative manner and underwent extensive application. An initial framework based on a scoping review was tested in rapid assessments, revealing inconsistencies with respect to the underlying concepts. Thus, pragmatic utility concept analysis was undertaken to advance the concepts of context and implementation. Based on these findings, the framework was revised and applied in several systematic reviews, one health technology assessment (HTA) and one applicability assessment of very different complex interventions. Lessons learnt from these applications and from peer review were incorporated, resulting in the CICI framework. RESULTS: The CICI framework comprises three dimensions-context, implementation and setting-which interact with one another and with the intervention dimension. Context comprises seven domains (i.e., geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political); implementation consists of five domains (i.e., implementation theory, process, strategies, agents and outcomes); setting refers to the specific physical location, in which the intervention is put into practise. The intervention and the way it is implemented in a given setting and context can occur on a micro, meso and macro level. Tools to operationalise the framework comprise a checklist, data extraction tools for qualitative and quantitative reviews and a consultation guide for applicability assessments. CONCLUSIONS: The CICI framework addresses and graphically presents context, implementation and setting in an integrated way. It aims at simplifying and structuring complexity in order to advance our understanding of whether and how interventions work. The framework can be applied in systematic reviews and HTA as well as primary research and facilitate communication among teams of researchers and with various stakeholders.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper presents the Context and Implementation of Complex Interventions (CICI) framework, which integrates context, implementation, and setting for complex interventions. It was developed iteratively through literature searches, concept analysis, and extensive application in systematic reviews, HTA, and applicability assessment.

Key findings

  • The CICI framework comprises three interacting dimensions: context, implementation, and setting. Context has seven domains (geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political); implementation has five domains (implementation theory, process, strategies, agents, outcomes); setting refers to the specific physical location where the intervention is put into practice.
  • Development included systematic searches yielding 35 publications on implementation and 17 on context, followed by pragmatic utility concept analysis that distinguished context, implementation, and setting.
  • The revised framework was applied in three rapid assessments, a demonstration HTA of reinforced home-based palliative care, quantitative and qualitative systematic reviews, and an applicability assessment with semi-structured consultations with palliative care experts from three European countries; it was also externally peer-reviewed by three experts.
  • Tools developed include a generic checklist, data extraction tools for qualitative and quantitative reviews, and a consultation guide; a worked example applied CICI, CFIR, and PARiHS to the Irish Air Pollution Act, finding CFIR and PARiHS had difficulty capturing public health context.

Methods

  • The CICI framework was developed in a three-step iterative process within the INTEGRATE-HTA project: initial framework from purposive literature searches and three rapid assessments; pragmatic utility concept analysis using systematic searches for context in EMBASE/MEDLINE and forward tracking for implementation in Google Scholar; revision and extensive application.
  • Applications included a demonstration HTA, quantitative systematic reviews, a qualitative systematic review using CICI as a best-fit framework, and an applicability assessment; appraisal used seven pre-specified criteria and external peer review.
  • A worked example applied CICI, CFIR, and PARiHS to the Air Pollution Act on the Marketing, Sale and Distribution of Coal, using the primary study and grey literature searches.

Limitations

  • The paper states that searches lacked explicit coverage of management and organisational studies; the setting dimension is less well conceptualised because the search strategy targeted context and implementation; no formal consensus method such as Delphi was used; and the latest framework version has not yet been fully tested.
  • The paper states that primary studies report little detail on context and implementation, limiting population of the framework, and that extracting data on context and implementation imposes additional workload in quantitative reviews, potentially substantial if further sources are consulted.
  • Evident from the text: the generic framework does not display relationships between dimensions or domains; applications noted overlaps between context and implementation dimensions and discomfort with the original ethical domain definition; peer reviewers questioned practical uptake given additional workload and called for clearer distinction between context and implementation and greater elaboration of domain definitions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s13012-017-0552-5

Cites

96 of 96 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Development and application of CICI framework
Fig. 1 Development and application of CICI frameworkCC BY · PubMed Central
Fig. 2 CICI framework. The context and implementation of complex interventions (CICI) framework comprises the three dimensions context, implementation and setting. The context comprises the seven domains: geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political context. Implementation consists of implementation theory, implementation process, implementation strategies, implementation agents and implementation outcomes. In the setting, the intervention and its implementation interact with the context. The shading of the semicircles illustrates the micro, meso and macro levels, on which implementation, context and setting can occur. Apart from the intervention of interest, the context and the way the intervention of interest is implemented may be advanced or compromised by other interventions occurring independently but targeting the same setting and population
Fig. 2 CICI framework. The context and implementation of complex interventions (CICI) framework comprises the three dimensions context, implementation and setting. The context comprises the seven domains: geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political context. Implementation consists of implementation theory, implementation process, implementation strategies, implementation agents and implementation outcomes. In the setting, the intervention and its implementation interact with the context. The shading of the semicircles illustrates the micro, meso and macro levels, on which implementation, context and setting can occur. Apart from the intervention of interest, the context and the way the intervention of interest is implemented may be advanced or compromised by other interventions occurring independently but targeting the same setting and populationCC BY · PubMed Central

Verification

Claims the paper makes

35 No basis given · 24 Not checked yet · 20 Own result · 19 Own interpretation · 7 Own method · 1 Partly supported · accuracy not yet measured

Verify all 42 references — free account

Show all 29 claims

  • Probably not a substantial claim. 50
  • It may state two separate claims as one. 46
77 of 77
  • The paper relies on this without support.

    Many existing frameworks, theories and models are specific rather than generic, e.g., focussing only on organisational context, and therefore do not lend themselves to application across diverse health interventions.

    “Moreover, many of these frameworks, theories and models are specific rather than generic, e.g., focussing only on organisational context, and therefore do not lend themselves to application across diverse health interventions.”
  • The paper relies on this without support.

    Few existing frameworks appear suitable for use within systematic reviews and HTAs.

    “Furthermore, few of these frameworks appear suitable for use within systematic reviews and HTAs.”
  • The paper relies on this without support.

    Many existing frameworks, models and theories are relatively abstract in nature and lack guidance on how to operationalise them in practise.

    “Indeed, many of these frameworks, models and theories are relatively abstract in nature and lack guidance on how to operationalise them in practise.”
  • The paper relies on this without support.

    Context, implementation and the intervention interact with one another and often co-evolve over time in a setting.

    “In the setting dimension, context, implementation and the intervention, which are described according to intervention theory, design and delivery characteristics [46], interact with one another and often co-evolve over time.”
  • The paper relies on this without support.

    Interventions are typically implemented simultaneously within different settings and at multiple levels.

    “Interventions are typically implemented simultaneously within different settings and at multiple levels.”
  • The paper relies on this without support.

    The ethical, socio-economic or epidemiological context have a considerable impact on the uptake, reach and effectiveness of an intervention.

    “Moreover, the ethical, socio-economic or epidemiological context, concepts that are regularly used in HTA and are known to have a considerable impact on the uptake, reach and effectiveness of an intervention, are rarely considered.”
  • The paper relies on this without support.

    An intervention and its implementation interact in multiple ways within any given setting and context, with these interactions taking place at multiple levels.

    “As stated repeatedly throughout this paper, an intervention and its implementation interact in multiple ways within any given setting and context, with these interactions taking place at multiple levels.”
  • The paper relies on this without support.

    The success in populating the dimensions and domains of the CICI framework with evidence critically depends on detailed assessment and reporting of information in primary studies.

    “The success in populating the dimensions and domains of the CICI framework with evidence critically depends on detailed assessment and reporting of information in primary studies.”
  • The paper relies on this without support.

    Many widely used reporting guidelines acknowledge context but most only require researchers to report on the setting.

    “While many of the widely used reporting guidelines, such as STROBE, SQUIRE, TREND and the extension of CONSORT for Pragmatic Trials, acknowledge context, most only require researchers to report on the setting.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 99 references

Research works printed but not linked: 13

  1. Medical Research Council. A framework for development and evaluation of RCTs for complex interventions to improve health. London: Medical Research Council; 2000.
  2. Fineout-Overholt E, Melnyk B. Organizational Culture and Readiness for System-Wide Implementation of EBP (OCR-SIEP) scale. Gilbert: ARCC llc Publishing; 2006.
  3. Burns J, Polus S, Brereton L, Ward S, Chilcott J, Pfadenhauer L, Rehfuess E. Effectiveness of home-based and reinforced home-based palliative care: a systematic review with semi-structured expert consultations. in submission process 2016.
  4. Rohwer A, Pfadenhauer LM, Burns J, Brereton L, Gerhardus A, Booth A, Oortwijn W, Rehfuess EA: Logic models help make sense of complexity in systematic reviews and health technology assessments. J Clin Epidemiol. 2016, in press. · DOI · Paper page
  5. Fellin P. Understanding American communities. In: Rothamn J, Erlich J, Tropman J, editors. Strategies of community organizations. Itasca: Peacock; 1995.
  6. Scott W. Conceptualizing organizational fields: Linking organizations and societal systems. In: Derlien H, Gerhardt U, Scharpf F, editors. Systemrationalitat und Partial Interesse [System Rationality and Partial Interests]. Baden Baden: Nomos Verlagsgesellschaft; 1994. p. 203–21.
  7. Bandura A. Social foundations of thought and action: a social cognitive theory. Prentice Hill: Englewood Cliffs, New Jersey; 1986.
  8. Rogers EM. Diffusion of innovations. 4th ed. New York: Free Press; 1995.
  9. Mozygemba K, Hofmann B, Lysdahl KB, Pfadenhauer L, Van Der Wilt GJ, Gerhardus A. Guidance to assess socio-cultural aspects. In: Lysdahl KB, Mozygemba K, Burns J, Chilcott JB, Brönneke JB, Hofmann B, editors. Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies. 2015.
  10. Statens Beredning för Medicinsk Utvärdering (SBU). Utvärdering av metoder i hälso- og sjukvården, en handbok. Etiska och sociala aspekter. Stockholm: SBU (Statens beredning för medicinsk utvärdering); 2014.
  11. Brönneke JB, Hofmann B, Lysdahl KB, Van Der Wilt GJ, Buchner B. Guidance to assess legal aspects. In: Lysdahl KB, Mozygemba K, Burns J, Chilcott JB, Brönneke JB, Hofmann B, editors. Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies. 2015.
  12. Metz A, Bartley L. Active implementation frameworks for program success. Zero to One. 2012;32:11–8.
  13. Peters M, Harmsen M, Laurant M, Wensing M. Ruimte voor verandering? Knelpunten en mogelijkheden voor verbeteringen in de patiëntenzorg [Room for improvement? Barriers to and facilitators for improvement of patient care]. Nijmegen: Centre for Quality of Care Research (WOK), Radboud University Nijmegen Medical Centre; 2002.

Cited by

Sorted by citations

Items 401–500 of 1,000

Metadata

Authors
Pfadenhauer, L. M., Gerhardus, A., Mozygemba, K., Lysdahl, K. B., Booth, A., Hofmann, B., Wahlster, P., Polus, S., Burns, J., Brereton, L., & Rehfuess, E.
Affiliations
  • Ludwig-Maximilians-Universität München (DE)
  • University of Bremen (DE)
  • University of Oslo (NO)
  • University of Sheffield (GB)
Corresponding author
Pfadenhauer, Lisa M.
Venue
Implementation Science
Published
2017-02-15
Volume
12
Issue
1
Pages
21-21
ISSN
1748-5908 (ISSN-L)
OpenAlex
W2588392437
Cited by
1,016
References
99
Citations by year
2026: 91 · 2025: 112 · 2024: 153 · 2023: 108 · 2022: 148 · 2021: 125 · 2020: 151 · 2019: 66 · 2018: 45 · 2017: 15
FWCI
13.33
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Implementation Science (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
  • Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion)
  • NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
  • Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
Funders
Seventh Framework Programme (306141)
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Public Administration › Evaluation and Performance Assessment
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
complex interventions, context, implementation, conceptual framework, implementation theory, setting, applicability assessment, health technology assessment, intervention effectiveness, stakeholder communication
MeSH terms
Bioethics, Diffusion of Innovation, Health Policy, Legislation as Topic, Socioeconomic Factors, Technology Assessment, Biomedical, Organizational Culture, Checklist

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review