Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework

Classified as probably not a substantial claim. May combine two claims.

The initial framework comprised two dimensions-context and implementation-with eight domains (locational, geographical, epidemiological, socio-cultural, socio-economic, ethical, legal and political) and four domains (provider, organisation and structure, funding and policy).

“The initial framework comprised two dimensions—context and implementation—with eight domains (i.e., locational, geographical, epidemiological, socio-cultural, socio-economic, ethical, legal and political) and four domains (i.e., provider, organisation and structure, funding and policy), respectively.”

May combine two claims.

The CICI framework was considered feasible to apply across all the different applications; ease of use was ensured through clear terminology, step-by-step guidance and ready-to-use data extraction tools.

“The framework was considered feasible to apply across all the different applications; ease of use was ensured through clear use of terminology, step-by-step guidance and the provision of ready-to-use data extraction tools.”

Classified as probably not a substantial claim. May combine two claims.

The CICI framework addresses and graphically presents context, implementation and setting in an integrated way.

“The CICI framework addresses and graphically presents context, implementation and setting in an integrated way.”

May combine two claims.

In the qualitative systematic review, the framework facilitated the assessment of relationships through data extracts being attributed to several domains; for example, 'access to healthcare' emerged as a relevant aspect under three context domains (geographical, political, ethical and socio-economic).

“On the other hand, in the qualitative systematic review, the framework facilitated the assessment of relationships through data extracts being attributed to several domains, for example, ‘access to healthcare’ emerged as a relevant aspect under three context domains (geographical, political, ethical and socio-economic).”

May combine two claims.

The framework was appraised as largely consistent, with significant overlaps between the context and implementation dimensions noted.

“In terms of coherence, the framework was appraised as largely consistent; significant overlaps between the context and implementation dimensions were noted.”

May combine two claims.

The additional time required for extracting data on context and implementation is limited when extraction is only performed on the included study, but may be substantial if further sources are also consulted.

“The additional time required is limited, where extraction is only performed on the included study; it may be substantial if further sources (e.g., qualitative studies or process evaluations cited in the included study) are also consulted.”

May combine two claims.

Based on systematic searches, 35 publications developing, proposing or describing theories, models or conceptual frameworks for implementation and 17 publications doing the same for context were included.

“Based on the systematic searches, we included 35 publications developing, proposing or describing theories, models or conceptual frameworks for implementation (see Additional file 1) and 17 publications doing the same for context (see Additional file 2) [37].”

• cites [37] Pfadenhauer LM, Mozygemba K, Gerhardus A, Hofmann B, Booth A, Lysdahl KB, Tummers M, Burns J, Rehfuess EA. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes. 2015;109:103–14. sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The framework was considered helpful in guiding the formulation of questions about links between domains and dimensions, which inspires advancement of theory regarding interactions.

“The framework was also considered helpful in guiding the formulation of questions about these links, which in return inspires the advancement of theory regarding interactions between domains as well as context and implementation dimensions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The forward tracking search based on Damschroder et al. (2009) resulted in considerably improved efficiency of searches without a significant loss of information.

“This resulted in considerably improved efficiency of our searches without a significant loss of information [37].”

• cites [37] Pfadenhauer LM, Mozygemba K, Gerhardus A, Hofmann B, Booth A, Lysdahl KB, Tummers M, Burns J, Rehfuess EA. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes. 2015;109:103–14. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The CICI framework has proven methodologically flexible, applicable in quantitative and qualitative systematic reviews and primary studies, though data extraction was sometimes limited by poor reporting.

“Generally, the framework has proven to be methodologically flexible due to its good applicability in quantitative and qualitative systematic reviews and the primary qualitative, quantitative or mixed-method studies included in these, although data extraction was sometimes limited by poor reporting.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In quantitative reviews, extracting data on context and implementation imposes an additional workload on researchers.

“In quantitative reviews, extracting data on context and implementation imposes an additional workload on researchers.”

May combine two claims.

The high granularity of the CICI framework facilitated extraction of often scarcely reported information on context and implementation and supported structuring this information.

“The high granularity of the framework proved capable of facilitating extraction of—often scarcely—reported information on context and implementation and supported the structuring of this information.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Data extractors were not comfortable with the original definition of the ethical domain.

“Also, data extractors were not comfortable with our original definition of the ethical domain.”

Classified as probably not a substantial claim.

Context comprises seven domains: geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political.

“Context comprises seven domains (i.e., geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political); implementation consists of five domains (i.e., implementation theory, process, strategies, agents and outcomes); setting refers to the specific physical location, in which the intervention is put into practise.”

Classified as probably not a substantial claim.

The concept of context is less studied and applied in diverse ways across different fields, among them implementation science.

“The concept of context, on the other hand, is less studied and applied in diverse ways across different fields, among them implementation science.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The concept of implementation is situated within an advancing and well-connected field of implementation science.

“The concept of implementation is situated within an advancing and well-connected field of implementation science.”

Classified as probably not a substantial claim. May combine two claims.

The concept analysis led to a clear conceptual distinction between context and setting as well as a more comprehensive formulation of characteristics for each of the domains of context and implementation.

“The concept analysis led to a clear conceptual distinction between context and setting as well as a more comprehensive formulation of characteristics for each of the domains of context and implementation.”

Classified as probably not a substantial claim. May combine two claims.

The CICI framework comprises three dimensions-context, implementation and setting-which interact with one another and with the intervention dimension.

“The CICI framework comprises three dimensions—context, implementation and setting—which interact with one another and with the intervention dimension.”

Classified as probably not a substantial claim.

A certain degree of adaptation with regard to the domains to be considered in relation to a specific intervention is a strength of the framework.

“A certain degree of adaptation with regard to the domains to be considered in relation to a specific intervention is clearly a strength of the framework.”

Classified as probably not a substantial claim.

Setting emerged as an additional and distinct concept of interest, sometimes used synonymously with context.

“Setting—a term that is sometimes used synonymously with context—emerged as an additional and distinct concept of interest.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In the qualitative review, issues arose in relation to the attribution of text extracts to specific domains (e.g., provider vs. organisational domain).

“In the qualitative review, issues arose in relation to the attribution of text extracts to specific domains (e.g., provider vs. organisational domain).”

Classified as probably not a substantial claim.

Implementation consists of five domains: implementation theory, process, strategies, agents and outcomes.

“Context comprises seven domains (i.e., geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political); implementation consists of five domains (i.e., implementation theory, process, strategies, agents and outcomes); setting refers to the specific physical location, in which the intervention is put into practise.”

Classified as probably not a substantial claim. May combine two claims.

While its generic version does not display relationships, the framework allows for their assessment when applied to a complex intervention.

“While its generic version does not display relationships, the framework allows for their assessment when applied to a complex intervention.”

May combine two claims.

Studies looking at policy and environmental interventions reported more details on the macro context, while studies evaluating educational, psychoeducational, psychological, social as well as spiritual interventions reported more details on the meso level relating to context or setting.

“While studies looking at policy and environmental interventions reported more details on the macro context, studies evaluating educational, psychoeducational, psychological, social as well as spiritual interventions reported more details on the meso level relating to context or setting.”

Classified as probably not a substantial claim.

The intervention, informed by theory, lies at the heart of the CICI framework with its reach and effectiveness being affected by context and implementation.

“The intervention, informed by theory, lies at the heart of the CICI framework with its reach and effectiveness being affected by context and implementation.”

Probably not a substantial claim: may not pass S4 (based).

The intervention and the way it is implemented in a given setting and context can occur on a micro, meso and macro level.

“The intervention and the way it is implemented in a given setting and context can occur on a micro, meso and macro level.”

Classified as probably not a substantial claim. May combine two claims.

Tools to operationalise the framework comprise a checklist, data extraction tools for qualitative and quantitative reviews and a consultation guide for applicability assessments.

“Tools to operationalise the framework comprise a checklist, data extraction tools for qualitative and quantitative reviews and a consultation guide for applicability assessments.”

May combine two claims.

The revised framework showed good internal coherence and completeness, applicability and user-friendliness and was found to contribute to theory advancement and development.

“Overall, as summarised in Table 3, the revised framework showed good internal coherence and completeness, applicability and user-friendliness and was found to contribute to theory advancement and development.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The graphical display arising from the framework's use highlights how dimensions and domains interact within a complex system.

“Moreover, the graphical display arising from its use highlights how dimensions and domains interact within a complex system.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Researchers considered the framework complete at the level of the domains but not with respect to the aspects covered.

“In relation to completeness, researchers considered the framework complete at the level of the domains but not with respect to the aspects covered.”

May combine two claims.

Through systematic literature searches and concept analysis, the CICI framework builds on previous knowledge and presents a strong theoretical basis; through empirical applications across several distinct complex health interventions, it has proven its value across a range of different methodological approaches.

“Through systematic literature searches and concept analysis, the CICI framework builds on previous knowledge and presents a strong theoretical basis; through empirical applications across several distinct complex health interventions, it has proven its value across a range of different methodological approaches (e.g., qualitative and quantitative systematic reviews, primary qualitative research).”

Probably not a substantial claim: may not pass S4 (based).

Numerous frameworks, theories and models assessing implementation and, to a lesser extent, context have been published over the past decades.

“Numerous frameworks, theories and models assessing implementation and, to a lesser extent, context have been published over the past decades.”

May combine two claims.

Issues identified in scientific articles and grey literature documents could easily be categorised under relevant domains, and the CICI framework facilitated explicit consideration of potential interactions occurring between domains and dimensions and across various levels.

“Issues identified in scientific articles and grey literature documents could easily be categorised under relevant domains, and the CICI framework also facilitated explicit consideration of potential interactions occurring between domains and dimensions and across various levels.”

Probably not a substantial claim: may not pass S4 (based).

The ethical, socio-economic or epidemiological context, concepts that are regularly used in HTA and are known to have a considerable impact on the uptake, reach and effectiveness of an intervention, are rarely considered.

“Moreover, the ethical, socio-economic or epidemiological context, concepts that are regularly used in HTA and are known to have a considerable impact on the uptake, reach and effectiveness of an intervention, are rarely considered.”

• AssumptionRests on an assumption: The ethical, socio-economic or epidemiological context have a considerable impact on the uptake, reach and effectiveness of an intervention.

Classified as probably not a substantial claim. May combine two claims.

All of these settings involve different implementation agents, require different implementation strategies and allow the assessment of different implementation outcomes.

“All of these settings involve different implementation agents, require different implementation strategies and allow the assessment of different implementation outcomes.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The CICI framework includes step-by-step pragmatic guidance, a generic checklist to be adapted for different purposes and data extraction tools as well as a graphical representation to facilitate its application in practise.

“The CICI framework includes step-by-step pragmatic guidance, a generic checklist to be adapted for different purposes and data extraction tools as well as a graphical representation to facilitate its application in practise.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Few of these frameworks address implementation and context in an integrated fashion, and very few offer pragmatic guidance and worked examples of how to apply the framework, theory or model in practise.

“Few of these frameworks address implementation and context in an integrated fashion, and very few offer pragmatic guidance and worked examples of how to apply the framework, theory or model in practise.”

May combine two claims.

As we applied the CFIR and PARiHS to the Air Pollution Act, many relevant aspects of context and implementation were not captured at all, while a substantial proportion of the CFIR and PARiHS domains remained unpopulated.

“Consequently, as we applied the CFIR and PARiHS to the Air Pollution Act, many relevant aspects of context and implementation were not captured at all, while a substantial proportion of the CFIR and PARiHS domains remained unpopulated.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

According to the feedback received, the CICI framework performed well against seven pre-specified criteria.

“In summary, according to the feedback received, the framework performed well against seven pre-specified criteria.”

May combine two claims.

We had difficulties with applying the CFIR and PARiHS to our intervention of choice, as both assume that those delivering and those receiving the intervention are nested within the organisation implementing the change; consequently, the detailed assessment of context exclusively relates to this organisational context.

“In contrast, we had difficulties with applying the CFIR and PARiHS to our intervention of choice, as both assume that those delivering and those receiving the intervention (e.g., doctors, nurses) are nested within the organisation implementing the change (e.g., hospitals, health centres); consequently, the detailed assessment of context exclusively relates to this organisational context.”

May combine two claims.

Previously published theories, frameworks or models, as identified from our systematic searches, are based on different theoretical underpinnings, although these are often not made explicit.

“Previously published theories, frameworks or models, as identified from our systematic searches are based on different theoretical underpinnings, although these are often not made explicit.”

May combine two claims.

The effectiveness of complex interventions, and their success in reaching all relevant target populations, is critically influenced by their implementation in a given context; effectiveness, implementation and context are inextricably linked.

“The effectiveness of complex interventions, as well as their success in reaching all relevant target populations, is critically influenced by their implementation in a given context; indeed, effectiveness, implementation and context are inextricably linked [1, 2].”

• cites [1] Waters E, Hall BJ, Armstrong R, Doyle J, Pettman TL, de Silva-Sanigorski A. Essential components of public health evidence reviews: capturing intervention complexity, implementation, economics and equity. J Public Health (Oxf). 2011;33:462–5. sourceNot checked yet

  Not checked in the free run.

• cites [2] Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13:95. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The framework can be applied in systematic reviews and HTA as well as primary research and facilitate communication among teams of researchers and with various stakeholders.

“The framework can be applied in systematic reviews and HTA as well as primary research and facilitate communication among teams of researchers and with various stakeholders.”

May combine two claims.

An implementation effort can be promoted and facilitated by single implementation leaders and/or implementation teams.

“An implementation effort can be promoted and facilitated by single implementation leaders [14, 15] and/or implementation teams [82].”

• cites [14] Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4. sourceNot checked yet

  Not checked in the free run.

• cites [15] Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of Innovations in Service Organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629. sourceNot checked yet

  Not checked in the free run.

• cites [82] Metz A, Bartley L. Active implementation frameworks for program success. Zero to One. 2012;32:11–8.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The CICI framework is likely to be equally relevant for primary research.

“However, given that primary research, evidence synthesis and evidence-based policy and practise are inextricably linked [32], the framework is likely to be equally relevant for primary research.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The searches had not been designed to capture all relevant aspects, which precluded a formal concept analysis.

“Our searches had not been designed to capture all relevant aspects, which precluded a formal concept analysis.”

Classified as probably not a substantial claim.

Implementation is considered to be a process, a constellation of processes, efforts or the means or methods of fitting, assimilating or putting into use an intervention-either evidence-based or theory-based-in an organisation or a setting.

“Implementation is considered to be a process, a constellation of processes, efforts or the means or methods of fitting, assimilating or putting into use an intervention—either evidence-based or theory-based—in an organisation or a setting [14, 30, 31].”

• cites [14] Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4. sourceNot checked yet

  Not checked in the free run.

• cites [30] Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. A glossary for dissemination and implementation research in health. J Public Health Manag Pract. 2008;14:117–23. sourceNot checked yet

  Not checked in the free run.

• cites [31] Simpson DD. A framework for implementing sustainable oral health promotion interventions. J Public Health Dent. 2011;71:S84–94. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The framework was externally peer-reviewed by three experts with intentions to apply the framework.

“At the same time, the framework was externally peer-reviewed by three experts with intentions to apply the framework.”

May combine two claims.

The CICI framework helps to review all sources of complexity in relation to a given intervention and to examine the interactions between them.

“It therefore helps to review all sources of complexity in relation to a given intervention and to examine the interactions between them.”

May combine two claims.

Uses of the terms 'context' and 'implementation' and their meanings vary widely in the health literature, lacking consensual definitions and descriptions as well as clearly delineated boundaries.

“Importantly, uses of the terms ‘context’ and ‘implementation’ and their meanings vary widely in the health literature, lacking consensual definitions and descriptions as well as clearly delineated boundaries.”

• cites [13] Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Quality in Health Care. 1998;7:149–58. sourceNot checked yet

  Not checked in the free run.

• cites [28] McCormack B, Kitson A, Harvey G, Rycroft-Malone J, Titchen A, Seers K. Getting evidence into practice: the meaning of ‘context’. J Adv Nurs. 2002;38:94–104. sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Context is often used synonymously with setting and environment, embracing static and dynamic aspects as well as the theory underpinning the intervention and its implementation.

“Context is often used synonymously with setting and environment [13, 28], embracing static (e.g., physical environment) and dynamic aspects (e.g., relationships, networks) as well as the theory underpinning the intervention and its implementation [29].”

• cites [13] Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Quality in Health Care. 1998;7:149–58. sourceNot checked yet

  Not checked in the free run.

• cites [28] McCormack B, Kitson A, Harvey G, Rycroft-Malone J, Titchen A, Seers K. Getting evidence into practice: the meaning of ‘context’. J Adv Nurs. 2002;38:94–104. sourceNot checked yet

  Not checked in the free run.

• cites [29] Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10 Suppl 1:21–34. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Context and implementation are rarely assessed in an integrated way, although selected frameworks have attempted to do so.

“The two aspects are rarely assessed in an integrated way, although selected frameworks have attempted to do so (e.g., [14, 27]).”

• cites [14] Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4. sourceNot checked yet

  Not checked in the free run.

• cites [27] Stetler CB, Damschroder LJ, Helfrich CD, Hagedorn HJ. A guide for applying a revised version of the PARIHS framework for implementation. Implement Sci. 2011;6. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

These other interventions are likely to influence the reach and effectiveness of the intervention under investigation in a specific setting.

“These other interventions are likely to influence the reach and effectiveness of the intervention under investigation in a specific setting.”

Probably not a substantial claim: may not pass S2 (checkable).

Operationalised as a checklist and/or graphical representation, the CICI framework can help researchers and research organisations to communicate with the policy and practise audiences they intend to reach.

“Operationalised as a checklist and/or graphical representation, the framework can help researchers and research organisations to communicate with the policy and practise audiences they intend to reach.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

McCormack and colleagues looked at context as part of the PARiHS framework; Squires and colleagues have not yet published their concept analysis.

“McCormack and colleagues looked at context as part of the Promoting Action on Research Implementation in Health Services (PARiHS) framework [37]; Squires and colleagues have not yet published their concept analysis [90].”

• cites [37] Pfadenhauer LM, Mozygemba K, Gerhardus A, Hofmann B, Booth A, Lysdahl KB, Tummers M, Burns J, Rehfuess EA. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes. 2015;109:103–14. sourceNot checked yet

  Not checked in the free run.

• cites [90] Squires JE, Graham ID, Hutchinson AM, Linklater S, Brehaut JC, Curran J, Ivers N, Lavis JN, Michie S, Sales AE, et al. Understanding context in knowledge translation: a concept analysis study protocol. J Adv Nurs. 2015;71:1146–55. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Previously published approaches primarily focus on implementation, although the details of implementation theory, implementation strategy or a comprehensive implementation process are often neglected.

“Previously published approaches primarily focus on implementation, although the details of implementation theory, implementation strategy or a comprehensive implementation process are often neglected (e.g., [14, 53, 84, 85]).”

• cites [14] Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4. sourceNot checked yet

  Not checked in the free run.

• cites [53] Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38:4–23. sourceNot checked yet

  Not checked in the free run.

• cites [84] May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, Rapley T, Ballini L, Ong BN, Rogers A, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4. sourceNot checked yet

  Not checked in the free run.

• cites [85] May C. Towards a general theory of implementation. Implement Sci. 2013;8. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Integrating insights gained in previously published frameworks, theories or models, the CICI framework adds a macro perspective that seeks to capture all aspects of complex interventions.

“Integrating insights gained in previously published frameworks, theories or models, it adds a macro perspective that seeks to capture all aspects of complex interventions.”

Classified as probably not a substantial claim. May combine two claims.

The CICI framework highlights the breadth and depth of the influence of context from a societal perspective and shows that context can act at one or several different levels (micro, meso and macro), thereby adding granularity and flexibility to the assessment of a complex intervention.

“The CICI framework highlights this breadth and depth of the influence of context from a societal perspective and shows that context can act at one or several different levels (micro, meso and macro), thereby adding granularity and flexibility to the assessment of a complex intervention.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors did not employ any formal method of consensus building, such as a Delphi approach, in developing the CICI framework.

“Importantly, we did not employ any formal method of consensus building, such as a Delphi approach.”

Probably not a substantial claim: may not pass S3 (load-bearing).

We are aware of at least two other concept analyses of context.

“We are aware of at least two other concept analyses of context [90].”

• cites [90] Squires JE, Graham ID, Hutchinson AM, Linklater S, Brehaut JC, Curran J, Ivers N, Lavis JN, Michie S, Sales AE, et al. Understanding context in knowledge translation: a concept analysis study protocol. J Adv Nurs. 2015;71:1146–55. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Context often plays a subordinate role; where it is considered explicitly, it is usually restricted to an organisational context.

“Context often plays a subordinate role; where it is considered explicitly, it is usually restricted to an organisational context [27, 44, 91].”

• cites [27] Stetler CB, Damschroder LJ, Helfrich CD, Hagedorn HJ. A guide for applying a revised version of the PARIHS framework for implementation. Implement Sci. 2011;6. sourceNot checked yet

  Not checked in the free run.

• cites [44] Kitson AL, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implementation science : IS. 2008;3:1–1. sourceNot checked yet

  Not checked in the free run.

• cites [91] Rycroft-Malone J, Seers K, Chandler J, Hawkes CA, Crichton N, Allen C, Bullock I, Strunin L. The role of evidence, context, and facilitation in an implementation trial: implications for the development of the PARIHS framework. Implement Sci. 2013;8. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

It is impossible to look for and discover everything.

“while keeping in mind that it is impossible to look for and discover everything [99].”

• cites [99] Peters M, Harmsen M, Laurant M, Wensing M. Ruimte voor verandering? Knelpunten en mogelijkheden voor verbeteringen in de patiëntenzorg [Room for improvement? Barriers to and facilitators for improvement of patient care]. Nijmegen: Centre for Quality of Care Research (WOK), Radboud University Nijmegen Medical Centre; 2002.Not checked yet

  Not checked in the free run.

May combine two claims.

For public health/complex interventions, there is often an organisational or even sectoral disconnection between the implementing organisation(s) and those receiving the intervention, and context is concerned with the society at large.

“Instead, for public health/complex interventions, there is often an organisational or even sectoral disconnection between the implementing organisation(s) and those receiving the intervention, and context is concerned with the society at large.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The CICI framework pays tribute to competing interventions that might have a considerable impact on the uptake, reach and effectiveness of the intervention being investigated.

“Lastly, the CICI framework pays tribute to competing interventions that might have a considerable impact on the uptake, reach and effectiveness of the intervention being investigated.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The setting dimension of the CICI framework is less well conceptualised, mostly because the search strategy was designed to yield insights into the concepts of context and implementation.

“It must also be noted that the setting dimension is less well conceptualised, mostly because our search strategy was designed to yield insights into the concepts of context and implementation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A limitation of the searches is the lack of explicit searches in databases of management and organisational studies.

“Some limitations pertaining to the searches are described elsewhere, for example the lack of explicit searches in databases of management and organisational studies [37].”

• cites [37] Pfadenhauer LM, Mozygemba K, Gerhardus A, Hofmann B, Booth A, Lysdahl KB, Tummers M, Burns J, Rehfuess EA. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes. 2015;109:103–14. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The literature tends to conceptualise setting as having physical boundaries, which may fall short of acknowledging developments during the last decade such as telemedicine and Internet- or smart phone-based interventions.

“While the literature tends to conceptualise setting as having physical boundaries [98], this may fall short of acknowledging developments during the last decade (e.g., telemedicine, Internet- or smart phone-based interventions).”

• cites [98] World Health Organization, Division of Health Promotion Education and Communication. Health promotion glossary. Geneva: World Health Organization; 1998.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

The CICI framework constitutes one way of structuring this complexity in order to advance our understanding of whether and how interventions work.

“The CICI framework constitutes one way of structuring this complexity in order to advance our understanding of whether and how interventions work, while keeping in mind that it is impossible to look for and discover everything [99].”

Probably not a substantial claim: may not pass S2 (checkable).

In moving beyond the organisational context, the framework is particularly suitable for public health and other interventions that are assessed or analysed on a macro level or on several levels concurrently.

“In moving beyond the organisational context, the framework is particularly suitable for public health and other interventions that are assessed or analysed on a macro level or on several levels concurrently.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Complexity is not only inherent in the intervention but also a consequence of interactions between the intervention and its implementation in context.

“Complexity is not only inherent in the intervention but also a consequence of interactions between the intervention and its implementation in context.”

• AssumptionRests on an assumption: An intervention and its implementation interact in multiple ways within any given setting and context, with these interactions taking place at multiple levels.

May combine two claims.

The CICI framework can serve both as a determinant framework that seeks to conceptualise, describe and understand the multiple influences on implementation outcomes, and as an evaluation framework that clarifies the context, setting and implementation aspects to be assessed when examining implementation success or lack thereof.

“It can serve both as a determinant framework that seeks to conceptualise, describe and understand the multiple influences on implementation outcomes, and as an evaluation framework that clarifies the context, setting and implementation aspects to be assessed when examining implementation success or lack thereof.”

May combine two claims.

To our knowledge, the CICI framework is the first attempt to provide guidance for assessing setting, context and implementation of complex interventions in a comprehensive manner, for integrating the views of different disciplines and for adapting a concept of context that is embedded in a broad public health perspective rather than a narrower organisational perspective.

“To our knowledge, the CICI framework is the first attempt to provide guidance for assessing setting, context and implementation of complex interventions in a comprehensive manner, for integrating the views of different disciplines (economics, ethics, sociology, law) and for adapting a concept of context that is embedded in a broad public health perspective rather than a narrower organisational”

Probably not a substantial claim: may not pass S3 (load-bearing).

The latest version of the CICI framework has not yet been fully tested.

“A limitation is, however, that the latest version of the framework has not yet been fully tested.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Over the last few decades, considerable work has been undertaken to elaborate and define elements of organisational context as well as behaviours that influence an implementation effort.

“Over the last few decades, considerable work has been undertaken to elaborate and define elements of organisational context as well as behaviours that influence an implementation effort (REFS).”

• cites REFSNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

The CICI framework can serve as a comprehensive starting point for examining context and implementation; where useful, granularity within specific domains can be added by bringing in insights from previously published frameworks.

“In this way, the CICI framework can serve as a comprehensive starting point for examining context and implementation; where useful, granularity within specific domains can be added by bringing in insights from previously published frameworks.”

May combine two claims.

While developed primarily for evidence synthesis, the usefulness of the CICI framework depends on the evidence available.

“Finally, while developed primarily for evidence synthesis, the usefulness of the framework depends on the evidence available.”

Classified as probably not a substantial claim. May combine two claims.

The combination of systematic searches to identify existing frameworks, theories and models of context and implementation and formal methods to derive definitions and detailed characteristics of the relevant concepts provides a strong theoretical basis for the CICI framework and constitutes one of its major strengths.

“The combination of systematic searches to identify existing frameworks, theories and models of context and implementation and formal methods to derive definitions and detailed characteristics of the relevant concepts provides a strong theoretical basis for the CICI framework and constitutes one of its major strengths.”