Methods for designing interventions to change healthcare professionals’ behaviour: a systematic review
Open accessAbstract
BACKGROUND: Systematic reviews consistently indicate that interventions to change healthcare professional (HCP) behaviour are haphazardly designed and poorly specified. Clarity about methods for designing and specifying interventions is needed. The objective of this review was to identify published methods for designing interventions to change HCP behaviour. METHODS: A search of MEDLINE, Embase, and PsycINFO was conducted from 1996 to April 2015. Using inclusion/exclusion criteria, a broad screen of abstracts by one rater was followed by a strict screen of full text for all potentially relevant papers by three raters. An inductive approach was first applied to the included studies to identify commonalities and differences between the descriptions of methods across the papers. Based on this process and knowledge of related literatures, we developed a data extraction framework that included, e.g. level of change (e.g. individual versus organization); context of development; a brief description of the method; tasks included in the method (e.g. barrier identification, component selection, use of theory). RESULTS: 3966 titles and abstracts and 64 full-text papers were screened to yield 15 papers included in the review, each outlining one design method. All of the papers reported methods developed within a specific context. Thirteen papers included barrier identification and 13 included linking barriers to intervention components; although not the same 13 papers. Thirteen papers targeted individual HCPs with only one paper targeting change across individual, organization, and system levels. The use of theory and user engagement were included in 13/15 and 13/15 papers, respectively. CONCLUSIONS: There is an agreement across methods of four tasks that need to be completed when designing individual-level interventions: identifying barriers, selecting intervention components, using theory, and engaging end-users. Methods also consist of further additional tasks. Examples of methods for designing the organisation and system-level interventions were limited. Further analysis of design tasks could facilitate the development of detailed guidelines for designing interventions.
AI snapshot
Written from the paper’s full text.
TL;DR
This systematic review identified 15 published methods for designing interventions to change healthcare professionals’ behaviour. Across these methods, four design tasks were commonly included: barrier identification, linking barriers to intervention components, using theory, and engaging end-users.
Key findings
- From 3966 screened citations and 64 full-text papers, 15 papers were included, each outlining one design method; methods were published between 2001 and 2014.
- 13/15 papers included barrier identification and 13/15 linked barriers to intervention components, though these were not the same 13 papers; 13/15 used theory and 13/15 included user engagement.
- 13/15 methods targeted individual healthcare professionals, while only one method targeted change across individual, organization, and system levels.
- Design tasks ranged from 2 to 7 steps, with a median of 5 tasks; 11/15 methods incorporated other published approaches, tools, or resources.
Methods
- Systematic review searching MEDLINE, Embase, and PsycINFO from 1996 to April 2015, with no published protocol; inclusion required a paper to describe a method for designing interventions to change healthcare professionals’ behaviour and focus primarily on the intervention design process.
- Title/abstract screening was done by one rater, followed by full-text screening by three raters independently; mean Kappa across all pairs was .43, indicating moderate agreement.
- Data extraction used a two-stage process: an inductive framework was developed by three reviewers, then data were extracted by two individuals independently followed by consensus discussion for discrepancies.
Limitations
- The paper states that only one rater screened titles and abstracts, which may have increased the possibility of omitting relevant studies; it also did not include books or book chapters, excluded studies that reported intervention testing alongside development, did not search grey literature, and used only three databases, making the review susceptible to publication bias.
- The paper states that all included methods were developed specifically for healthcare environments, and methods from other disciplines or before 1996 may have been missed.
- The paper notes that it did not measure the degree to which the identified methods are used; the text also makes evident that the search ended in April 2015 and excluded non-English articles.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 82 cited sources checked · 0 supported · 0 flagged
- Findingown data
Thirteen of the 15 included papers targeted individual healthcare professionals.
“Thirteen papers targeted individual HCPs with only one paper targeting change across individual, organization, and system levels.”
Quote in the paperNot a substantial claim - Findingnone given
The 13 papers that included barrier identification and the 13 that included linking barriers to intervention components were not the same 13 papers.
“Thirteen papers included barrier identification and 13 included linking barriers to intervention components; although not the same 13 papers.”
Quote in the paperNot a substantial claim - Findingown data
Use of theory was included in 13 of the 15 papers.
“The use of theory and user engagement were included in 13/15 and 13/15 papers, respectively.”
Quote in the paperNot a substantial claim - Findingown data
Thirteen of the 15 included papers included barrier identification.
“Thirteen papers included barrier identification and 13 included linking barriers to intervention components; although not the same 13 papers.”
Quote in the paperNot a substantial claim - Findingown data
All 15 included papers reported methods developed within a specific context.
“All of the papers reported methods developed within a specific context.”
Quote in the paperNot a substantial claim
- Findingown data
Only one included paper targeted change across individual, organization, and system levels.
“Thirteen papers targeted individual HCPs with only one paper targeting change across individual, organization, and system levels.”
Quote in the paperNot a substantial claim - Findingown data
Eleven of the 15 included papers incorporated other approaches, tools, or resources as a component of the intervention design process.
“Eleven of the 15 papers incorporated other approaches, tools, or resources as a component of the intervention design process [14, 19–28].”
Quote in the paperNot a substantial claim0 of 10 sources checked- cites [14] Curran GM, Mukherjee S, Allee E, Owen RR. A process for developing an implementation intervention: QUERI Series. Implement Sci. 2008;3(1):17. source
Not checked in the free run.
- cites [19] Taylor N, Lawton R, Slater B, Foy R. The demonstration of a theory-based approach to the design of localized patient safety interventions. Implement Sci. 2013;8(1):123. source
Not checked in the free run.
- cites [20] Van Bokhoven M, Kok G, Van der Weijden T. Designing a quality improvement intervention: a systematic approach. Qual Saf Health Care. 2003;12(3):215–20. source
Not checked in the free run.
- cites [21] French SD, Green SE, O’Connor DA, McKenzie JE, Francis JJ, Michie S, Buchbinder R, Schattner P, Spike N, Grimshaw JM. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7(1):38. source
Not checked in the free run.
- cites [22] Kolehmainen N, Francis JJ. Specifying content and mechanisms of change in interventions to change professionals’ practice: an illustration from the Good Goals study in occupational therapy. Implement Sci. 2012;7(1):100. source
Not checked in the free run.
- cites [23] Cabassa LJ, Druss B, Wang Y, Lewis-Fernández R. Collaborative planning approach to inform the implementation of a healthcare manager intervention for Hispanics with serious mental illness: a study protocol. Implement Sci. 2011;6:80. source
Not checked in the free run.
- cites [24] Clyne B, Bradley MC, Hughes CM, Clear D, McDonnell R, Williams D, Fahey T, Smith SM. Addressing potentially inappropriate prescribing in older patients: development and pilot study of an intervention in primary care (the OPTI-SCRIPT study). BMC Health Serv Res. 2013;13(1):307. source
Not checked in the free run.
- cites [25] Porcheret M, Main C, Croft P, McKinley R, Hassell A, Dziedzic K. Development of a behaviour change intervention: a case study on the practical application of theory. Implement Sci. 2014;9(1):42. source
Not checked in the free run.
- cites [26] Sassen B, Kok G, Mesters I, Crutzen R, Cremers A, Vanhees L. A web-based intervention for health professionals and patients to decrease cardiovascular risk attributable to physical inactivity: development process. JMIR Res Protocols. 2012;1(2):e21. source
Not checked in the free run.
- cites [27] Schmid AA, Andersen J, Kent T, Williams LS, Damush TM. Using intervention mapping to develop and adapt a secondary stroke prevention program in Veterans Health Administration medical centers. Implement Sci. 2010;5(1):11. source
Not checked in the free run.
- Findingown data
Examples of methods for designing organisation- and system-level interventions were limited.
“Examples of methods for designing the organisation and system-level interventions were limited.”
Quote in the paperSubstantial claim- Rests on an assumption: The search strategy and databases used were sufficient to identify all relevant published methods for designing interventions to change healthcare professionals' behaviour.
- Findingown data
Four of the 15 included papers incorporated Intervention Mapping.
“Four of these [20, 23, 26, 27] incorporated Intervention Mapping [29]”
Quote in the paperNot a substantial claim0 of 5 sources checked- cites [20] Van Bokhoven M, Kok G, Van der Weijden T. Designing a quality improvement intervention: a systematic approach. Qual Saf Health Care. 2003;12(3):215–20. source
Not checked in the free run.
- cites [23] Cabassa LJ, Druss B, Wang Y, Lewis-Fernández R. Collaborative planning approach to inform the implementation of a healthcare manager intervention for Hispanics with serious mental illness: a study protocol. Implement Sci. 2011;6:80. source
Not checked in the free run.
- cites [26] Sassen B, Kok G, Mesters I, Crutzen R, Cremers A, Vanhees L. A web-based intervention for health professionals and patients to decrease cardiovascular risk attributable to physical inactivity: development process. JMIR Res Protocols. 2012;1(2):e21. source
Not checked in the free run.
- cites [27] Schmid AA, Andersen J, Kent T, Williams LS, Damush TM. Using intervention mapping to develop and adapt a secondary stroke prevention program in Veterans Health Administration medical centers. Implement Sci. 2010;5(1):11. source
Not checked in the free run.
- cites [29] Bartholomew LK, Parcel GS, Kok G, Gottlieb NH. Planning health promotion programs: an intervention mapping approach. San Francisco: John Wiley & Sons; 2011.
Not checked in the free run.
- Findingown data
User engagement was included in 13 of the 15 papers.
“The use of theory and user engagement were included in 13/15 and 13/15 papers, respectively.”
Quote in the paperNot a substantial claim - Findingown data
Two of the 15 included papers incorporated the Behaviour Change Technique taxonomy without also incorporating the Theoretical Domains Framework.
“two incorporated the BCT taxonomy without also incorporating the TDF [22, 28]”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [22] Kolehmainen N, Francis JJ. Specifying content and mechanisms of change in interventions to change professionals’ practice: an illustration from the Good Goals study in occupational therapy. Implement Sci. 2012;7(1):100. source
Not checked in the free run.
- cites [28] Foy R, Francis JJ, Johnston M, Eccles M, Lecouturier J, Bamford C, Grimshaw J. The development of a theory-based intervention to promote appropriate disclosure of a diagnosis of dementia. BMC Health Serv Res. 2007;7:207. source
Not checked in the free run.
- Findingown data
Three of the 15 included papers incorporated both the Theoretical Domains Framework and the Behaviour Change Technique taxonomy.
“another three incorporated both the TDF and the BCT taxonomy [19, 21, 25]”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [19] Taylor N, Lawton R, Slater B, Foy R. The demonstration of a theory-based approach to the design of localized patient safety interventions. Implement Sci. 2013;8(1):123. source
Not checked in the free run.
- cites [21] French SD, Green SE, O’Connor DA, McKenzie JE, Francis JJ, Michie S, Buchbinder R, Schattner P, Spike N, Grimshaw JM. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7(1):38. source
Not checked in the free run.
- cites [25] Porcheret M, Main C, Croft P, McKinley R, Hassell A, Dziedzic K. Development of a behaviour change intervention: a case study on the practical application of theory. Implement Sci. 2014;9(1):42. source
Not checked in the free run.
- Findingown data
Six of the 15 papers used the structured interview processes outlined in the Theoretical Domains Framework or in Intervention Mapping.
“Six papers used the structured interview processes outlined in the TDF [19, 21, 25] or in Intervention Mapping [20, 26, 27].”
Quote in the paperNot a substantial claim0 of 6 sources checked- cites [19] Taylor N, Lawton R, Slater B, Foy R. The demonstration of a theory-based approach to the design of localized patient safety interventions. Implement Sci. 2013;8(1):123. source
Not checked in the free run.
- cites [21] French SD, Green SE, O’Connor DA, McKenzie JE, Francis JJ, Michie S, Buchbinder R, Schattner P, Spike N, Grimshaw JM. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7(1):38. source
Not checked in the free run.
- cites [25] Porcheret M, Main C, Croft P, McKinley R, Hassell A, Dziedzic K. Development of a behaviour change intervention: a case study on the practical application of theory. Implement Sci. 2014;9(1):42. source
Not checked in the free run.
- cites [20] Van Bokhoven M, Kok G, Van der Weijden T. Designing a quality improvement intervention: a systematic approach. Qual Saf Health Care. 2003;12(3):215–20. source
Not checked in the free run.
- cites [26] Sassen B, Kok G, Mesters I, Crutzen R, Cremers A, Vanhees L. A web-based intervention for health professionals and patients to decrease cardiovascular risk attributable to physical inactivity: development process. JMIR Res Protocols. 2012;1(2):e21. source
Not checked in the free run.
- cites [27] Schmid AA, Andersen J, Kent T, Williams LS, Damush TM. Using intervention mapping to develop and adapt a secondary stroke prevention program in Veterans Health Administration medical centers. Implement Sci. 2010;5(1):11. source
Not checked in the free run.
- Findingown data
Thirteen of the 15 papers included linking barriers to intervention components.
“All but two papers [23, 24] included linking barriers to intervention components.”
- Also stated in: § Discussion · plain
“Thirteen papers included some form of barrier identification and 13 provided direction for linking barriers to intervention components; however, these were not the same 13 papers.”
Quote in the paperNot a substantial claim - Also stated in: § Discussion · plain
- Findingown data
Across the 15 identified methods papers, the number of design tasks ranged from two to seven, with a median of 5.
“All of the 15 identified papers included a number of tasks required for the design process. These steps ranged from two [14] to seven [28] with a median of 5 tasks.”
- Also stated in: § Discussion · plain
“The methods consisted of at least two tasks and, at most, seven tasks.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [14] Curran GM, Mukherjee S, Allee E, Owen RR. A process for developing an implementation intervention: QUERI Series. Implement Sci. 2008;3(1):17. source
Not checked in the free run.
- cites [28] Foy R, Francis JJ, Johnston M, Eccles M, Lecouturier J, Bamford C, Grimshaw J. The development of a theory-based intervention to promote appropriate disclosure of a diagnosis of dementia. BMC Health Serv Res. 2007;7:207. source
Not checked in the free run.
- Also stated in: § Discussion · plain
- Findingown data
Thirteen of the 15 papers included some form of barrier identification.
“All but two [22, 23] of the papers included some form of barrier identification.”
- Also stated in: § Discussion · plain
“Thirteen papers included some form of barrier identification and 13 provided direction for linking barriers to intervention components; however, these were not the same 13 papers.”
Quote in the paperNot a substantial claim - Also stated in: § Discussion · plain
- Findingown data
Eleven of the 15 methods incorporated other tools or resources, albeit not in identical ways.
“The majority of the methods found (11/15) incorporated other tools or resources, albeit not in identical ways.”
Quote in the paperNot a substantial claim - Findingown data
The 13 papers that included barrier identification were not the same 13 papers that provided direction for linking barriers to intervention components.
“Thirteen papers included some form of barrier identification and 13 provided direction for linking barriers to intervention components; however, these were not the same 13 papers.”
Quote in the paperNot a substantial claim - Findingown data
In all cases where input was gathered, this involved testing, piloting, or showing the intervention to the targets and gathering feedback in the form of discussion or interview.
“In all cases, this involved testing, piloting, or showing the intervention to the targets and gathering feedback in the form of discussion or interview.”
Quote in the paperNot a substantial claim - Findingcitation
One study [22] targeted teams as well as individuals but did not do so at the organizational level.
“A second study [22] targeted teams as well as individuals but did not do so at the organizational level.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [22] Kolehmainen N, Francis JJ. Specifying content and mechanisms of change in interventions to change professionals’ practice: an illustration from the Good Goals study in occupational therapy. Implement Sci. 2012;7(1):100. source
Not checked in the free run.
- Findingown data
One of the 15 papers targeted change across individual, organization, and system levels.
“Thirteen papers targeted only individual HCPs with one paper targeting change across individual, organization, and system levels.”
Quote in the paperNot a substantial claim - Findingown data
Four of the 15 methods papers for designing interventions to change HCPs' behaviour incorporated Intervention Mapping.
“However, we did find four methods papers for designing interventions to change HCPs’ behaviour that incorporated Intervention Mapping [20, 23, 26, 27].”
Quote in the paperNot a substantial claim0 of 4 sources checked- cites [20] Van Bokhoven M, Kok G, Van der Weijden T. Designing a quality improvement intervention: a systematic approach. Qual Saf Health Care. 2003;12(3):215–20. source
Not checked in the free run.
- cites [23] Cabassa LJ, Druss B, Wang Y, Lewis-Fernández R. Collaborative planning approach to inform the implementation of a healthcare manager intervention for Hispanics with serious mental illness: a study protocol. Implement Sci. 2011;6:80. source
Not checked in the free run.
- cites [26] Sassen B, Kok G, Mesters I, Crutzen R, Cremers A, Vanhees L. A web-based intervention for health professionals and patients to decrease cardiovascular risk attributable to physical inactivity: development process. JMIR Res Protocols. 2012;1(2):e21. source
Not checked in the free run.
- cites [27] Schmid AA, Andersen J, Kent T, Williams LS, Damush TM. Using intervention mapping to develop and adapt a secondary stroke prevention program in Veterans Health Administration medical centers. Implement Sci. 2010;5(1):11. source
Not checked in the free run.
- Findingown data
All 15 papers reported methods that were developed within a specific context.
“All of the papers reported methods that were developed within a specific context.”
Quote in the paperNot a substantial claim - Findingcitation
A number of papers identified in the search stated intervention design as an aim but lacked the detail that would allow replication.
“In addition, a number of papers were identified that specifically stated intervention design as an aim [36, 37] but that lacked the detail that would allow replication.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [36] Flottorp S, Oxman AD. Identifying barriers and tailoring interventions to improve the management of urinary tract infections and sore throat: a pragmatic study using qualitative methods. BMC Health Serv Res. 2003;3(1):3. source
Not checked in the free run.
- cites [37] Forsetlund L, Bjørndal A. Identifying barriers to the use of research faced by public health physicians in Norway and developing an intervention to reduce them. J Health Serv Res Policy. 2002;7(1):10–8. source
Not checked in the free run.
- Findingown data
The systematic review found 15 papers that outlined 15 methods for designing interventions to change HCPs' behaviour.
“We found 15 papers that outlined 15 methods.”
- Also stated in: § Conclusions · plain
“This systematic review outlined 15 published and replicable methods for designing interventions to change HCPs’ behaviour.”
Quote in the paperSubstantial claim - Also stated in: § Conclusions · plain
- Findingown data
Thirteen of the 15 papers included the use of theory.
“All but two papers [24, 32] included the use of theory.”
- Also stated in: § Discussion · plain
“Thirteen papers included the use of theory, and another 13 included gathering input on the design of the intervention from the targets of the intervention.”
- Also stated in: § Discussion · plain
“Almost all of the methods found (13/15) used theory at some point in the tasks for intervention design, yet evidence indicates theory is rarely used in the design of interventions, or at least rarely reported [39, 40].”
Quote in the paperSubstantial claim - Also stated in: § Discussion · plain
- Findingown data
One of the 15 papers (Clyne et al. [24]) conducted a barriers assessment but did not describe how barriers were linked to intervention components.
“The second paper [24] did conduct a barriers assessment but did not describe how barriers were linked to intervention components.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [24] Clyne B, Bradley MC, Hughes CM, Clear D, McDonnell R, Williams D, Fahey T, Smith SM. Addressing potentially inappropriate prescribing in older patients: development and pilot study of an intervention in primary care (the OPTI-SCRIPT study). BMC Health Serv Res. 2013;13(1):307. source
Not checked in the free run.
- Findingown data
Other papers used more specific, discrete theories chosen based on the context in which the specific intervention was being developed.
“Other papers used more specific, discrete theories chosen based on the context in which the specific intervention was being developed.”
Quote in the paperNot a substantial claim - Findingown data
Thirteen of the 15 papers included some approach to gathering input on the design of the intervention from the users or target individuals.
“All but two papers [18, 30] included some approach to gathering input on the design of the intervention from the users or the individuals that were the target of the intervention.”
- Also stated in: § Discussion · plain
“Thirteen papers included the use of theory, and another 13 included gathering input on the design of the intervention from the targets of the intervention.”
Quote in the paperNot a substantial claim - Also stated in: § Discussion · plain
- Findingown data
Four steps appear common to intervention design: barrier identification, linking barriers to intervention component selection, use of theory, and user engagement.
“There appear to be four steps common to intervention design: barrier identification, linking barriers to intervention component selection, use of theory, and user engagement (i.e. seeking input on feasibility or acceptability of the intervention from the potential targets).”
- Also stated in: § Conclusions · hedged
“Although these methods included varied steps, there was a general agreement that designing an intervention for individual-level change includes identifying barriers, selecting intervention components, using theory, and engaging end-users.”
Quote in the paperSubstantial claim - Also stated in: § Conclusions · hedged
- Findingown data
Only one study [14] explicitly targeted change in organisations or systems.
“We found only one study [14] that did this explicitly.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [14] Curran GM, Mukherjee S, Allee E, Owen RR. A process for developing an implementation intervention: QUERI Series. Implement Sci. 2008;3(1):17. source
Not checked in the free run.
- Findingcitation
Some of the 15 papers used broad theoretical frameworks by incorporating pre-existing approaches that had used theory in their development, such as the Theoretical Domains Framework.
“There were papers (e.g. [21]) that used broad theoretical frameworks by way of incorporating pre-existing approaches that had used theory in their development (e.g. the TDF [17]).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [21] French SD, Green SE, O’Connor DA, McKenzie JE, Francis JJ, Michie S, Buchbinder R, Schattner P, Spike N, Grimshaw JM. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7(1):38. source
Not checked in the free run.
- cites [17] Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7(1):37. source
Not checked in the free run.
- Findingcitation
One of the 15 papers used 'development panels' involving staff, research experts, clinical experts, and local champions in meetings and consultations to link barriers to intervention components.
“One paper used what they termed ‘development panels’ which involved staff, research experts, clinical experts, and local champions participating in a range of meetings and consultations [14].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [14] Curran GM, Mukherjee S, Allee E, Owen RR. A process for developing an implementation intervention: QUERI Series. Implement Sci. 2008;3(1):17. source
Not checked in the free run.
- Findingown data
Thirteen of the 15 papers targeted only individual HCPs.
“Thirteen papers targeted only individual HCPs with one paper targeting change across individual, organization, and system levels.”
Quote in the paperNot a substantial claim - Interpretationown data
There is agreement across methods of four tasks that need to be completed when designing individual-level interventions: identifying barriers, selecting intervention components, using theory, and engaging end-users.
“There is an agreement across methods of four tasks that need to be completed when designing individual-level interventions: identifying barriers, selecting intervention components, using theory, and engaging end-users.”
Quote in the paperSubstantial claim- Rests on an assumption: The search strategy and databases used were sufficient to identify all relevant published methods for designing interventions to change healthcare professionals' behaviour.
- Rests on an assumption: The included papers' descriptions of methods were sufficiently detailed and representative to allow extraction of the common design tasks.
- Rests on an assumption: The descriptive variables selected for extraction were the most critical for understanding intervention design tasks.
- Interpretationnone given
Methods for designing interventions also consist of further additional tasks beyond the four agreed tasks.
“Methods also consist of further additional tasks.”
Quote in the paperSubstantial claim- Rests on an assumption: The descriptive variables selected for extraction were the most critical for understanding intervention design tasks.
- Premisecitation
Systematic reviews of knowledge translation interventions to change healthcare professionals' practice consistently indicate that interventions are haphazardly designed and poorly specified, limiting replication, understanding, and generalizability.
“Systematic reviews of KT interventions to change HCPs’ practice consistently indicate that interventions are haphazardly designed and poorly specified, limiting our ability for replication, understanding, and generalizability [5, 6].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [5] Flodgren G, Parmelli E, Doumit G, Gattellari M, O’Brien MA, Grimshaw J, Eccles MP. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2011;(8):CD000125. doi: 10.1002/14651858.CD000125.pub4 . source
Not checked in the free run.
- cites [6] Michie S, Johnston M. Theories and techniques of behaviour change: developing a cumulative science of behaviour change. Health Psychol Rev. 2012;6(1):1–6. source
Not checked in the free run.
- Premisereasoning
One issue contributing to the shortcomings in intervention design is a lack of agreed, practical 'how to' guidance for designing knowledge translation interventions.
“One issue contributing to the shortcomings in intervention design is a lack of agreed, practical ‘how to’ guidance for designing KT interventions.”
Quote in the paperSubstantial claim - Premisereasoning
Sources describing a range of complete methods for intervention design are few.
“but sources describing a range of complete methods for intervention design are few.”
Quote in the paperSubstantial claim - Premisecitation
Limitations in intervention design impede evaluations of interventions.
“Limitations in intervention design impede evaluations of interventions [7, 8].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [7] Gardner B, Whittington C, McAteer J, Eccles MP, Michie S. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Soc Sci Med. 2010;70(10):1618–1625. source
Not checked in the free run.
- cites [8] Van Hoof TJ, Miller NE, Meehan TP. Do published studies of educational outreach provide documentation of potentially important characteristics? Am J Med Qual. 2013;28(6):480–4. source
Not checked in the free run.
- Interpretationreasoning
Having two raters would have reduced the possibility of omitting a potentially relevant study.
“Although support exists for the validity of using one rater [41], having two raters would have reduced the possibility of omitting a potentially relevant study.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [41] Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R. Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med. 2002;21(11):1635–40. source
Not checked in the free run.
- Interpretationreasoning
The exclusion of studies that also reported testing was done in part to isolate methods described in enough detail to replicate and adequately guide intervention design.
“In part, we did this to isolate methods that were described in enough detail to be able to replicate and adequately guide the design of an intervention.”
Quote in the paperSubstantial claim - Study design and conductreasoning
Studies that reported on the testing of an intervention in addition to its development were not included.
“Our inclusion criteria meant that we did not include any studies that reported on the testing of an intervention in addition to the development of the intervention.”
Quote in the paperSubstantial claim - Interpretationreasoning
Selecting one of these published methods or building on these methods is likely to guide researchers to use theory.
“Selecting one of these published methods or building on these methods is likely to guide researchers to use theory.”
Quote in the paperNot a substantial claim
- Classified as probably not a substantial claim. 25
- May combine two claims. 18
Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).
Findingown dataThirteen of the 15 included papers included linking barriers to intervention components.
“Thirteen papers included barrier identification and 13 included linking barriers to intervention components; although not the same 13 papers.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe systematic review identified 15 papers, each outlining one design method for interventions to change healthcare professionals' behaviour.
“3966 titles and abstracts and 64 full-text papers were screened to yield 15 papers included in the review, each outlining one design method.”
Quote in the paperSubstantial claim- Rests on an assumption: The search strategy and databases used were sufficient to identify all relevant published methods for designing interventions to change healthcare professionals' behaviour.
May combine two claims.
Findingown dataAmong the papers that covered barrier identification, the methods used included observations, interviews and/or focus groups, surveys, literature reviews, structured reflection by researchers, job analysis and expert consensus, and predictive studies to identify factors influencing behaviour.
“In the papers where barrier identification was covered, the methods included observations, interviews and/or focus groups [14, 18, 30–32], surveys [32], literature reviews [24], structured reflection by the researchers [32], job analysis and expert consensus [18], and undertaking a predictive study to identify factors influencing the behaviour [28].”
Quote in the paperNot a substantial claim0 of 7 sources checked- cites [14] Curran GM, Mukherjee S, Allee E, Owen RR. A process for developing an implementation intervention: QUERI Series. Implement Sci. 2008;3(1):17. source
Not checked in the free run.
- cites [18] Battles J. Improving patient safety by instructional systems design. Qual Saf Health Care. 2006;15 suppl 1:i25–9. source
Not checked in the free run.
- cites [30] Chandler C, Meta J, Ponzo C, Nasuwa F, Kessy J, Mbakilwa H, Haaland A, Reyburn H. The development of effective behaviour change interventions to support the use of malaria rapid diagnostic tests by Tanzanian clinicians. Implement Sci. 2014;9:83. source
Not checked in the free run.
- cites [31] McDermott L, Yardley L, Little P, Ashworth M, Gulliford M. Developing a computer delivered, theory based intervention for guideline implementation in general practice. BMC Fam Pract. 2010;11(1):90. source
Not checked in the free run.
- cites [32] Fretheim A, Oxman AD, Flottorp S. Improving prescribing of antihypertensive and cholesterol-lowering drugs: a method for identifying and addressing barriers to change. BMC Health Serv Res. 2004;4(1):23. source
Not checked in the free run.
- cites [24] Clyne B, Bradley MC, Hughes CM, Clear D, McDonnell R, Williams D, Fahey T, Smith SM. Addressing potentially inappropriate prescribing in older patients: development and pilot study of an intervention in primary care (the OPTI-SCRIPT study). BMC Health Serv Res. 2013;13(1):307. source
Not checked in the free run.
- cites [28] Foy R, Francis JJ, Johnston M, Eccles M, Lecouturier J, Bamford C, Grimshaw J. The development of a theory-based intervention to promote appropriate disclosure of a diagnosis of dementia. BMC Health Serv Res. 2007;7:207. source
Not checked in the free run.
May combine two claims.
FindingcitationOne of the 15 papers (Kolehmainen et al. [22]) assumed barriers had already been identified in previous work and provided a method for linking barriers to intervention components.
“One of these papers [22] reported an assumption that barriers had already been identified in previous work and provided a method for linking barriers to intervention components.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [22] Kolehmainen N, Francis JJ. Specifying content and mechanisms of change in interventions to change professionals’ practice: an illustration from the Good Goals study in occupational therapy. Implement Sci. 2012;7(1):100. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataFive of the 15 included papers incorporated just one other published tool.
“Five of 15 incorporated just one other published tool [20, 24, 26–28]”
Quote in the paperNot a substantial claim0 of 5 sources checked- cites [20] Van Bokhoven M, Kok G, Van der Weijden T. Designing a quality improvement intervention: a systematic approach. Qual Saf Health Care. 2003;12(3):215–20. source
Not checked in the free run.
- cites [24] Clyne B, Bradley MC, Hughes CM, Clear D, McDonnell R, Williams D, Fahey T, Smith SM. Addressing potentially inappropriate prescribing in older patients: development and pilot study of an intervention in primary care (the OPTI-SCRIPT study). BMC Health Serv Res. 2013;13(1):307. source
Not checked in the free run.
- cites [26] Sassen B, Kok G, Mesters I, Crutzen R, Cremers A, Vanhees L. A web-based intervention for health professionals and patients to decrease cardiovascular risk attributable to physical inactivity: development process. JMIR Res Protocols. 2012;1(2):e21. source
Not checked in the free run.
- cites [27] Schmid AA, Andersen J, Kent T, Williams LS, Damush TM. Using intervention mapping to develop and adapt a secondary stroke prevention program in Veterans Health Administration medical centers. Implement Sci. 2010;5(1):11. source
Not checked in the free run.
- cites [28] Foy R, Francis JJ, Johnston M, Eccles M, Lecouturier J, Bamford C, Grimshaw J. The development of a theory-based intervention to promote appropriate disclosure of a diagnosis of dementia. BMC Health Serv Res. 2007;7:207. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationOne of the 15 included papers incorporated three published tools.
“and one incorporated three published tools [25].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [25] Porcheret M, Main C, Croft P, McKinley R, Hassell A, Dziedzic K. Development of a behaviour change intervention: a case study on the practical application of theory. Implement Sci. 2014;9(1):42. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataTwo of the 15 included papers incorporated the Medical Research Council (MRC) framework.
“and two incorporated the Medical Research Council (MRC) framework [22, 24].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [22] Kolehmainen N, Francis JJ. Specifying content and mechanisms of change in interventions to change professionals’ practice: an illustration from the Good Goals study in occupational therapy. Implement Sci. 2012;7(1):100. source
Not checked in the free run.
- cites [24] Clyne B, Bradley MC, Hughes CM, Clear D, McDonnell R, Williams D, Fahey T, Smith SM. Addressing potentially inappropriate prescribing in older patients: development and pilot study of an intervention in primary care (the OPTI-SCRIPT study). BMC Health Serv Res. 2013;13(1):307. source
Not checked in the free run.
May combine two claims.
Findingown dataOne of the 15 papers (Cabassa et al. [23]) focused on adapting an intervention using stakeholder engagement and did not address barrier identification specifically.
“The other paper [23] focused on adapting an intervention using stakeholder engagement and did not address barrier identification specifically.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23] Cabassa LJ, Druss B, Wang Y, Lewis-Fernández R. Collaborative planning approach to inform the implementation of a healthcare manager intervention for Hispanics with serious mental illness: a study protocol. Implement Sci. 2011;6:80. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationEccles et al. [38] includes a description of using a theory to design a KT intervention.
“One of these was Eccles et al. [38] which includes a description of using a theory to design a KT intervention.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [38] Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005;58(2):107–12. source
Not checked in the free run.
May combine two claims.
FindingcitationAmong the papers that linked barriers to intervention components, methods included mapping TDF barriers to the BCT taxonomy and using the structured approaches described in Intervention Mapping.
“Methods to link barriers to intervention components included mapping TDF-barriers to the BCT taxonomy [19, 21, 25], as well as using the structured approaches described in Intervention Mapping [20, 26, 27].”
Quote in the paperNot a substantial claim0 of 6 sources checked- cites [19] Taylor N, Lawton R, Slater B, Foy R. The demonstration of a theory-based approach to the design of localized patient safety interventions. Implement Sci. 2013;8(1):123. source
Not checked in the free run.
- cites [21] French SD, Green SE, O’Connor DA, McKenzie JE, Francis JJ, Michie S, Buchbinder R, Schattner P, Spike N, Grimshaw JM. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7(1):38. source
Not checked in the free run.
- cites [25] Porcheret M, Main C, Croft P, McKinley R, Hassell A, Dziedzic K. Development of a behaviour change intervention: a case study on the practical application of theory. Implement Sci. 2014;9(1):42. source
Not checked in the free run.
- cites [20] Van Bokhoven M, Kok G, Van der Weijden T. Designing a quality improvement intervention: a systematic approach. Qual Saf Health Care. 2003;12(3):215–20. source
Not checked in the free run.
- cites [26] Sassen B, Kok G, Mesters I, Crutzen R, Cremers A, Vanhees L. A web-based intervention for health professionals and patients to decrease cardiovascular risk attributable to physical inactivity: development process. JMIR Res Protocols. 2012;1(2):e21. source
Not checked in the free run.
- cites [27] Schmid AA, Andersen J, Kent T, Williams LS, Damush TM. Using intervention mapping to develop and adapt a secondary stroke prevention program in Veterans Health Administration medical centers. Implement Sci. 2010;5(1):11. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationIn two of the 15 papers, gathering input included formal cognitive interviews.
“In two cases, this included formal cognitive interviews [28, 31].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [28] Foy R, Francis JJ, Johnston M, Eccles M, Lecouturier J, Bamford C, Grimshaw J. The development of a theory-based intervention to promote appropriate disclosure of a diagnosis of dementia. BMC Health Serv Res. 2007;7:207. source
Not checked in the free run.
- cites [31] McDermott L, Yardley L, Little P, Ashworth M, Gulliford M. Developing a computer delivered, theory based intervention for guideline implementation in general practice. BMC Fam Pract. 2010;11(1):90. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
FindingcitationThe Intervention Mapping paper [29] outlined a method for intervention design for health behaviours, not for HCPs' behaviours, and was therefore excluded.
“This paper outlined a method for intervention design for health behaviours, not for HCPs’ behaviours, and was therefore excluded.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [29] Bartholomew LK, Parcel GS, Kok G, Gottlieb NH. Planning health promotion programs: an intervention mapping approach. San Francisco: John Wiley & Sons; 2011.
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenAll of the included methods were developed specifically for healthcare environments.
“All of the included methods were developed specifically for healthcare environments.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThere are limited methods that specifically take organisation and system level contexts into consideration, or that consider all levels.
“We found limited methods to specifically take the organisation and system level contexts into consideration, as well as methods that consider all levels.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataLimited methods target change in organisations or systems, or were developed with a focus on the organization or system.
“First, limited methods target change in organisations or systems, or at least were developed with a focus on the organization or system.”
- Also stated in: § Conclusions · plain
“Methods for designing organisation and system-level interventions were limited.”
Quote in the paperSubstantial claim- Also stated in: § Conclusions · plain
May combine two claims.
FindingcitationOne of the 15 papers (Cabassa et al. [23]) focused on adapting an intervention using stakeholder engagement and did not address linking barriers to intervention components specifically.
“As mentioned, one of these papers [23] focused on adapting an intervention using stakeholder engagement and did not address linking to intervention components specifically.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23] Cabassa LJ, Druss B, Wang Y, Lewis-Fernández R. Collaborative planning approach to inform the implementation of a healthcare manager intervention for Hispanics with serious mental illness: a study protocol. Implement Sci. 2011;6:80. source
Not checked in the free run.
May combine two claims.
FindingcitationExamples of specific theories used included Social Cognitive Theory to design a computer-delivered intervention to enhance use of practice guidelines in general practices, and theories of risk perception to improve physical activity in cardiovascular patients as part of Intervention Mapping.
“Examples included using Social Cognitive Theory [33] to design a computer-delivered intervention to enhance the use of practice guidelines in general practices [31], and using theories of risk perception to improve physical activity in cardiovascular patients as part of the Intervention Mapping process [26].”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [33] Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191. source
Not checked in the free run.
- cites [31] McDermott L, Yardley L, Little P, Ashworth M, Gulliford M. Developing a computer delivered, theory based intervention for guideline implementation in general practice. BMC Fam Pract. 2010;11(1):90. source
Not checked in the free run.
- cites [26] Sassen B, Kok G, Mesters I, Crutzen R, Cremers A, Vanhees L. A web-based intervention for health professionals and patients to decrease cardiovascular risk attributable to physical inactivity: development process. JMIR Res Protocols. 2012;1(2):e21. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
FindingcitationEccles et al. [38] provided a rationale and description of conceptual issues related to using theory to develop an intervention, but the degree of detail on how to design an intervention is limited.
“We felt this paper provided a rationale and description of conceptual issues related to using theory to develop an intervention; however, the degree of detail on how to design an intervention in this paper is limited.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [38] Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005;58(2):107–12. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenAn interrater reliability analysis using the Kappa statistic was performed to determine consistency among raters for the full-text review.
“An interrater reliability analysis using the Kappa statistic was performed to determine consistency among raters for the full-text review.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThe review did not publish a protocol.
“We did not publish a protocol.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenDue to resource limitations, articles not in English were excluded.
“Due to resource limitations, we also excluded articles that were not in English.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataA screen of titles and abstracts was conducted by one rater independently and was followed by a review of full papers by three raters independently.
“A screen of titles and abstracts was conducted by one rater independently (shared by HLC, JES, NK) and was followed by a review of full papers by three raters independently (HLC, JES, NK).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenPapers were included if they described a method for designing interventions to change healthcare professionals' behaviour or practice and the primary focus was on the intervention design process.
“Papers were included if two criteria were met: (1) the paper described a method (process, tasks, approach) for designing interventions to change HCPs’ behaviour or practice, and (2) the primary focus of the paper was on the intervention design process (as opposed to, e.g. on intervention evaluation).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataData extraction was conducted by two individuals independently first followed by consensus discussions for discrepancies.
“Data extraction was conducted by two individuals (HLC, JES) independently first followed by consensus discussions for discrepancies (Stage 2).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductcitationThe mean Kappa statistic across all pairs was .43, indicating moderate agreement.
“The mean Kappa statistic across all pairs was .43 indicated moderate agreement [16].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [16] Landis JR, Koch GG The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThree reviewers progressed in iterative cycles of reading and discussing included papers to identify similarities and differences and develop extraction items until agreement.
“Three reviewers (HLC, JES, NK) progressed in iterative cycles of reading and discussing the included papers to identify similarities and differences between them and used these discussions to develop a list of items to be extracted. The iterative cycles were continued until an agreement between the three authors was reached.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductreasoningPapers were excluded if they lacked enough detail for replication.
“we excluded papers that lacked enough detail for replication.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationcitationAlthough many methods' approaches to barrier identification could result in a focus on the organisation or system if barriers at these levels are identified, the implicit focus of these papers targeted individual behaviour change.
“While it is true that many of the methods’ approaches to barrier identification could result in a focus on the organisation or system should barriers at these levels be identified (for example, see French et al. [21]), the implicit focus of these papers targeted individual behaviour change.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [21] French SD, Green SE, O’Connor DA, McKenzie JE, Francis JJ, Michie S, Buchbinder R, Schattner P, Spike N, Grimshaw JM. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7(1):38. source
Not checked in the free run.
May combine two claims.
PremisecitationEvidence indicates that theory is rarely used in the design of interventions, or at least rarely reported.
“Almost all of the methods found (13/15) used theory at some point in the tasks for intervention design, yet evidence indicates theory is rarely used in the design of interventions, or at least rarely reported [39, 40].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [39] Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, Carroll K, Chalifoux M, Eva KW. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8:66. source
Not checked in the free run.
- cites [40] Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010;5(14):5908–5.
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductreasoningBooks and book chapters were not included in the search due to challenges in adequately searching books.
“Due to the challenges in adequately searching books, we did not include books or book chapters in our search.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningIt is likely that other methods to design interventions to change health behaviours could similarly be adopted to design interventions to change HCPs' practice.
“It is likely that other methods to design interventions to change health behaviours could similarly be adopted to design interventions to change HCPs’ practice.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductcitationGrey literature was not searched, making the review susceptible to publication bias.
“Lastly, we did not search grey literature, making the review susceptible to publication bias [42], and we only chose to use three databases.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [42] Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9. source
Not checked in the free run.
Classified as probably not a substantial claim.
InterpretationreasoningThe four common steps represent a simple structure of potential prototypical steps for the design of a KT intervention.
“While we do not necessarily understand the best order for these tasks nor do we know what additional tasks are required, it does represent a simple structure of potential prototypical steps for the design of a KT intervention.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenOnly one rater was used for the title and abstract review.
“We used only one rater for the title and abstract review.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningThere is a need for a better understanding of what activities constitute intervention design.
“The process of undertaking this review highlights a second gap: the need for a better understanding of what activities constitute intervention design.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThe review did not measure the degree to which these methods are used.
“Our review did not measure the degree to which these methods are used but this would be a useful future area of research.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenOnly three databases were used.
“Lastly, we did not search grey literature, making the review susceptible to publication bias [42], and we only chose to use three databases.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
The search strategy and databases used were sufficient to identify all relevant published methods for designing interventions to change healthcare professionals' behaviour.
“A sensitive search strategy was designed in consultation with an information science specialist (CF) using both subject headings and text terms and comprised a combination of three facets: professional behaviour change; theory, framework or technique; and interventions.”
The paper relies on this without support.
The included papers' descriptions of methods were sufficiently detailed and representative to allow extraction of the common design tasks.
“we excluded papers that lacked enough detail for replication.”
The paper relies on this without support.
The descriptive variables selected for extraction were the most critical for understanding intervention design tasks.
“The resulting descriptive variables to be extracted were believed to be the most critical, and it included a brief description of the design tasks.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Colquhoun and colleagues found 15 articles with replicable methods for designing strategies to change healthcare professionals' behavior.
“Colquhoun and colleagues (56) found 15 articles with replicable methods for designing strategies to change healthcare professionals' behavior, and Powell et al. (55) proposed Intervention Mapping (58), concept mapping (59), conjoint analysis (60), and system dynamics modeling (61) as methods to aid the design, selection, and tailoring of strategies.”
From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Van Bokhoven publication year differs between table and reference
MinorTable 1 dates the van Bokhoven paper to 2004, while its cited reference dates it to 2003.
“15. van Bokhoven 2004 [20]”
“20. Van Bokhoven M Kok G Van der Weijden T Designing a quality improvement intervention: a systematic approach Qual Saf Health Care 2003 12 3 215 20”
The denominator for named methods is 12 rather than 15
MinorThe paper describes 15 included papers, but uses 12 as the denominator when reporting how many gave their method a formal name.
“Three of the 12 papers reported a formal name or label for the method”
“Table 1 provides a summary of the included 15 papers that were published between 2001and 2014.”
Cabassa publication year differs between table and reference
MinorTable 1 dates the Cabassa paper to 2001, while its cited reference dates it to 2011.
“2. Cabassa, 2001[23]”
“23. Cabassa LJ Druss B Wang Y Lewis-Fernández R Collaborative planning approach to inform the implementation of a healthcare manager intervention for Hispanics with serious mental illness: a study protocol Implement Sci 2011 6 80”
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859 · see citing works
Self-efficacy: Toward a unifying theory of behavioral change
Bandura, A. (1977) · Psychological Review · doi:10.1037/0033-295x.84.2.191 · Cited by 43,925 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554 · see citing works
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769 · see citing works
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · see citing works
Self-efficacy: Toward a unifying theory of behavioral change
Bandura, A. (1978) · Advances in Behaviour Research and Therapy · doi:10.1016/0146-6402(78)90002-4 · Cited by 10,586 · see citing works
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548 · see citing works
Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J., & Wood, C. E. (2013) · Annals of Behavioral Medicine · doi:10.1007/s12160-013-9486-6 · Cited by 7,629 · see citing works
Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · see citing works
Audit and feedback: effects on professional practice and healthcare outcomes
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., O'Brien, M. A., Johansen, M., Grimshaw, J., & Oxman, A. D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000259.pub3 · Cited by 5,485 · see citing works
Self-citationCane, J., O’Connor, D., & Michie, S. (2012) · Implementation Science · doi:10.1186/1748-5908-7-37 · Cited by 4,888 · cited 9× · see citing works
“There were papers (e.g. [21]) that used broad theoretical frameworks by way of incorporating pre-existing approaches that had used theory in their development (e.g. the TDF [17]).”
BackgroundInfluentialLost in knowledge translation: Time for a map?
Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J., Caswell, W., & Robinson, N. (2006) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.47 · Cited by 4,705 · see citing works
Yeh, M. (2014) · Journal of Nursing Research · doi:10.1097/jnr.0000000000000065 · Cited by 3,836 · see citing works
Knowledge translation of research findings
Grimshaw, J. M., Eccles, M. P., Lavis, J. N., Hill, S. J., & Squires, J. E. (2012) · Implementation Science · doi:10.1186/1748-5908-7-50 · Cited by 2,407 · see citing works
Self-citationBero, L. A., Grilli, R., Grimshaw, J. M., Harvey, E., Oxman, A. D., & Thomson, M. A. (1998) · BMJ · doi:10.1136/bmj.317.7156.465 · Cited by 2,396 · see citing works
Self-citationPlanning health promotion programs an intervention mapping approach
Bartholomew, L. K., Parcel, G. S., Kok, G., & Gottlieb, N. (2006) · Cited by 2,038 · see citing works
Canadian Institutes of Health Research
Cooney-Birch, J. (2019) · Palgrave Macmillan UK eBooks · doi:10.1057/978-1-349-95943-3_219 · Cited by 1,457 · see citing works
French, S. D., Green, S. E., O’Connor, D. A., McKenzie, J. E., Francis, J. J., Michie, S., Buchbinder, R., Schattner, P., Spike, N., & Grimshaw, J. M. (2012) · Implementation Science · doi:10.1186/1748-5908-7-38 · Cited by 1,338 · see citing works
Self-citationFlottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M., Baker, R., & Eccles, M. P. (2013) · Implementation Science · doi:10.1186/1748-5908-8-35 · Cited by 1,138 · see citing works
Local opinion leaders: effects on professional practice and health care outcomes
Flodgren, G., Parmelli, E., Doumit, G., Gattellari, M., O'Brien, M. A., Grimshaw, J., & Eccles, M. P. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000125.pub4 · Cited by 1,070 · see citing works
Self-citationWhat counts as evidence in evidence‐based practice?
Rycroft‐Malone, J., Seers, K., Titchen, A., Harvey, G., Kitson, A. A., & McCormack, B. G. (2004) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2004.03068.x · Cited by 960 · see citing works
Eccles, M., Grimshaw, J., Walker, A., Johnston, M., & Pitts, N. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.09.002 · Cited by 917 · see citing works
Self-citationKnowledge Translation in Health Care: Moving from Evidence to Practice
(2009) · International Journal of Health Care Quality Assurance · doi:10.1108/ijhcqa.2009.06222fae.002 · Cited by 689 · see citing works
Davies, P., Walker, A. E., & Grimshaw, J. M. (2010) · Implementation Science · doi:10.1186/1748-5908-5-14 · Cited by 619 · see citing works
Self-citationTheories and techniques of behaviour change: Developing a cumulative science of behaviour change
Michie, S., & Johnston, M. (2012) · Health Psychology Review · doi:10.1080/17437199.2012.654964 · Cited by 522 · see citing works
Knowledge Translation in Health Care
(2013) · Wiley eBooks · doi:10.1002/9781118413555 · Cited by 480 · see citing works
Edwards, P., Clarke, M., DiGuiseppi, C., Pratap, S., Roberts, I., & Wentz, R. (2002) · Statistics in Medicine · doi:10.1002/sim.1190 · Cited by 340 · see citing works
A systematic review of the use of theory in randomized controlled trials of audit and feedback
Colquhoun, H. L., Brehaut, J. C., Sales, A., Ivers, N., Grimshaw, J., Michie, S., Carroll, K., Chalifoux, M., & Eva, K. W. (2013) · Implementation Science · doi:10.1186/1748-5908-8-66 · Cited by 231 · see citing works
Self-citationGardner, B., Whittington, C., McAteer, J., Eccles, M. P., & Michie, S. (2010) · Social Science & Medicine · doi:10.1016/j.socscimed.2010.01.039 · Cited by 225 · see citing works
Designing a quality improvement intervention: a systematic approach
van Bokhoven, M. A. (2003) · Quality and Safety in Health Care · doi:10.1136/qhc.12.3.215 · Cited by 204 · see citing works
A process for developing an implementation intervention: QUERI Series
Curran, G. M., Mukherjee, S., Allee, E., & Owen, R. R. (2008) · Implementation Science · doi:10.1186/1748-5908-3-17 · Cited by 110 · see citing works
The demonstration of a theory-based approach to the design of localized patient safety interventions
Taylor, N., Lawton, R., Slater, B., & Foy, R. (2013) · Implementation Science · doi:10.1186/1748-5908-8-123 · Cited by 77 · see citing works
Development of a behaviour change intervention: a case study on the practical application of theory
Porcheret, M., Main, C., Croft, P., McKinley, R., Hassell, A., & Dziedzic, K. (2014) · Implementation Science · doi:10.1186/1748-5908-9-42 · Cited by 72 · see citing works
Foy, R., Francis, J. J., Johnston, M., Eccles, M., Lecouturier, J., Bamford, C., & Grimshaw, J. (2007) · BMC Health Services Research · doi:10.1186/1472-6963-7-207 · Cited by 66 · see citing works
Self-citationFlottorp, S., & Oxman, A. D. (2003) · BMC Health Services Research · doi:10.1186/1472-6963-3-3 · Cited by 62 · see citing works
the eCRT Research Team, McDermott, L., Yardley, L., Little, P., Ashworth, M., & Gulliford, M. (2010) · BMC Family Practice · doi:10.1186/1471-2296-11-90 · Cited by 62 · see citing works
Clyne, B., Bradley, M. C., Hughes, C. M., Clear, D., McDonnell, R., Williams, D., Fahey, T., & Smith, S. M. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-307 · Cited by 58 · see citing works
Cabassa, L. J., Druss, B., Wang, Y., & Lewis-Fernández, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-80 · Cited by 50 · see citing works
Schmid, A. A., Andersen, J., Kent, T., Williams, L. S., & Damush, T. M. (2010) · Implementation Science · doi:10.1186/1748-5908-5-97 · Cited by 46 · see citing works
Trends in publications regarding evidence-practice gaps: A literature review
Evensen, A. E., Sanson‐Fisher, R., D’Este, C. A., & Fitzgerald, M. (2010) · Implementation Science · doi:10.1186/1748-5908-5-11 · Cited by 44 · see citing works
Improving patient safety by instructional systems design
Battles, J. B. (2006) · Quality in Health Care · doi:10.1136/qshc.2005.015917 · Cited by 40 · see citing works
Forsetlund, L., & Bjørndal, A. (2002) · Journal of Health Services Research & Policy · doi:10.1258/1355819021927629 · Cited by 33 · see citing works
Chandler, C. I. R., Meta, J., Ponzo, C., Nasuwa, F., Kessy, J., Mbakilwa, H., Haaland, A., & Reyburn, H. (2014) · Implementation Science · doi:10.1186/1748-5908-9-83 · Cited by 32 · see citing works
Sassen, B., Kok, G., Mesters, I., Crutzen, R., Cremers, A., & Vanhees, L. (2012) · JMIR Research Protocols · doi:10.2196/resprot.1804 · Cited by 26 · see citing works
Fretheim, A., Oxman, A. D., & Flottorp, S. (2004) · BMC Health Services Research · doi:10.1186/1472-6963-4-23 · Cited by 26 · see citing works
Kolehmainen, N., & Francis, J. J. (2012) · Implementation Science · doi:10.1186/1748-5908-7-100 · Cited by 24 · see citing works
Self-citationVan Hoof, T. J., Miller, N. E., & Meehan, T. P. (2013) · American Journal of Medical Quality · doi:10.1177/1062860613476335 · Cited by 9 · see citing works
Flagged first, then by citations
2 of the 50 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Waltz, T. J., Powell, B. J., Fernández, M. E., Abadie, B., & Damschroder, L. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0892-4 · Cited by 1,136 · see citing works
Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda
Powell, B. J., Fernandez, M. E., Williams, N. J., Aarons, G. A., Beidas, R. S., Lewis, C. C., McHugh, S. M., & Weiner, B. J. (2019) · Frontiers in Public Health · doi:10.3389/fpubh.2019.00003 · Cited by 775 · see citing works
Taxonomy of approaches to developing interventions to improve health: a systematic methods overview
O’Cathain, A., Croot, L., Sworn, K., Duncan, E., Rousseau, N. S., Turner, K. M., Yardley, L., & Hoddinott, P. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0425-6 · Cited by 581 · see citing works
Forsetlund, L., O’Brien, M. A., Forsén, L., Mwai, L., Reinar, L. M., Okwen, P., Horsley, T., & Rose, C. J. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003030.pub3 · Cited by 293 · see citing works
Designing and undertaking randomised implementation trials: guide for researchers
Wolfenden, L., Foy, R., Presseau, J., Grimshaw, J., Ivers, N. M., Powell, B. J., Taljaard, M., Wiggers, J., Sutherland, R., Nathan, N., et al. (2021) · BMJ · doi:10.1136/bmj.m3721 · Cited by 279 · see citing works
Self-citationThe effects of care bundles on patient outcomes: a systematic review and meta-analysis
Lavallée, J. F., Gray, T. A., Dumville, J. C., Russell, W., & Cullum, N. A. (2017) · Implementation Science · doi:10.1186/s13012-017-0670-0 · Cited by 262 · see citing works
Beidas, R. S., Dorsey, S., Lewis, C. C., Lyon, A. R., Powell, B. J., Purtle, J., Saldana, L., Shelton, R. C., Stirman, S. W., & Lane-Fall, M. B. (2022) · Implementation Science · doi:10.1186/s13012-022-01226-3 · Cited by 230 · see citing works
Leveraging Implementation Science to Improve Cancer Care Delivery and Patient Outcomes
Mitchell, S. A., & Chambers, D. (2017) · Journal of Oncology Practice · doi:10.1200/jop.2017.024729 · Cited by 116 · see citing works
Laban, G., Kappas, A., Morrison, V., & Cross, E. S. (2024) · International Journal of Social Robotics · doi:10.1007/s12369-023-01076-z · Cited by 119 · see citing works
Lavallée, J. F., Gray, T. A., Dumville, J. C., & Cullum, N. A. (2018) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2017.12.015 · Cited by 93 · see citing works
Tell me more! assessing interactions with social robots from speech
Laban, G., George, J.-N., Morrison, V., & Cross, E. S. (2020) · Paladyn, Journal of Behavioral Robotics · doi:10.1515/pjbr-2021-0011 · Cited by 81 · see citing works
Mangurian, C. V., Niu, G. C., Schillinger, D., Newcomer, J. W., Dilley, J., & Handley, M. A. (2017) · Implementation Science · doi:10.1186/s13012-017-0663-z · Cited by 75 · see citing works
Smith, J., Corace, K., MacDonald, T. K., Fabrigar, L. R., Saedi, A., Chaplin, A., MacFarlane, S., Valickis, D., & Garber, G. E. (2018) · Journal of Hospital Infection · doi:10.1016/j.jhin.2018.12.014 · Cited by 69 · see citing works
McHugh, S., Presseau, J., Luecking, C. T., & Powell, B. J. (2022) · Implementation Science · doi:10.1186/s13012-022-01227-2 · Cited by 64 · see citing works
Turner, K. M., Rousseau, N. S., Croot, L., Duncan, E., Yardley, L., O’Cathain, A., & Hoddinott, P. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028756 · Cited by 62 · see citing works
Balis, L. E., & Houghtaling, B. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00532-1 · Cited by 58 · see citing works
Conceptual tensions and practical trade-offs in tailoring implementation interventions
McHugh, S., Riordan, F., Curran, G. M., Lewis, C. C., Wolfenden, L., Presseau, J., Lengnick‐Hall, R., & Powell, B. J. (2022) · Frontiers in Health Services · doi:10.3389/frhs.2022.974095 · Cited by 56 · see citing works
Sharing Our Emotions With Robots: Why Do We Do It and How Does It Make Us Feel?
Laban, G., & Cross, E. S. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3470984 · Cited by 56 · see citing works
Croot, L., O’Cathain, A., Sworn, K., Yardley, L., Turner, K. M., Duncan, E., & Hoddinott, P. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0512-8 · Cited by 53 · see citing works
Lovero, K. L., Kemp, C. G., Wagenaar, B. H., Giusto, A., Greene, M. C., Powell, B. J., & Proctor, E. K. (2023) · Implementation Science · doi:10.1186/s13012-023-01310-2 · Cited by 52 · see citing works
Powell, B. J., Haley, A. D., Patel, S. V., Amaya-Jackson, L. M., Glienke, B., Blythe, M., Lengnick‐Hall, R., McCrary, S., Beidas, R. S., Lewis, C. C., et al. (2020) · Implementation Science Communications · doi:10.1186/s43058-020-00009-5 · Cited by 50 · see citing works
Parmar, J., Anderson, S. D., Duggleby, W., Holroyd-Leduc, J. M., Pollard, C., & Brémault‐Phillips, S. (2020) · Health & Social Care in the Community · doi:10.1111/hsc.13173 · Cited by 50 · see citing works
Matsuo, M. (2021) · Personnel Review · doi:10.1108/pr-01-2021-0026 · Cited by 49 · see citing works
Gordon, E. J., Lee, J. J., Kang, R. H., Caicedo, J. C., Holl, J. L., Ladner, D. P., & Shumate, M. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3151-5 · Cited by 48 · see citing works
Bird, M., Mortenson, W. B., Chu, F., Acerra, N., Bagnall, E., Wright, A., Hayley, K., Yao, J. K., & Eng, J. J. (2019) · Physical Therapy · doi:10.1093/ptj/pzy146 · Cited by 47 · see citing works
Learning from scaling up ultra-rapid genomic testing for critically ill children to a national level
Best, S., Brown, H. L., Lunke, S., Patel, C., Pinner, J. R., Barnett, C., Wilson, M., Sandaradura, S. A., McClaren, B., Brett, G. R., et al. (2021) · npj Genomic Medicine · doi:10.1038/s41525-020-00168-3 · Cited by 47 · see citing works
Moore, J. L., Virva, R., Henderson, C. E., Lenca, L., Butzer, J. F., Lovell, L. L., Roth, E. J., Graham, I. D., & Hornby, T. G. (2020) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2020.10.133 · Cited by 45 · see citing works
Evaluation of a Peer-to-Peer Data Transparency Intervention for Mohs Micrographic Surgery Overuse
Albertini, J. G., Wang, P., Fahim, C., Hutfless, S. M., Stasko, T., Vidimos, A. T., Leshin, B., Billingsley, E. M., Coldiron, B. M., Bennett, R. G., et al. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.1259 · Cited by 44 · see citing works
Broder‐Fingert, S., Carter, A. S., Pierce, K., Stone, W. L., Wetherby, A. M., Scheldrick, C., Smith, C., Bacon, E. C., James, S., Ibañez, L. V., et al. (2018) · Autism · doi:10.1177/1362361318766238 · Cited by 42 · see citing works
Keller, H., Valaitis, R. F., Laur, C. V., McNicholl, T., Xu, Y., Dubin, J. A., Curtis, L., Obiorah, S., Ray, S., Bernier, P., et al. (2018) · Clinical Nutrition · doi:10.1016/j.clnu.2018.02.009 · Cited by 41 · see citing works
Musa, S., Skrijelj, V., Kulo, A., Habersaat, K. B., Smjecanin, M., Primorac, E., Becirovic, D., & Jackson, C. (2020) · Vaccine · doi:10.1016/j.vaccine.2020.01.025 · Cited by 41 · see citing works
Bunger, A. C., Chuang, E., Girth, A. M., Lancaster, K. E., Smith, R., Phillips, R. J., Martin, J. K., Gadel, F., Willauer, T., Himmeger, M., et al. (2024) · Implementation Science · doi:10.1186/s13012-024-01335-1 · Cited by 40 · see citing works
Campbell, W. N., & Douglas, N. (2017) · Evidence-Based Communication Assessment and Intervention · doi:10.1080/17489539.2017.1370215 · Cited by 40 · see citing works
Fontaine, G., Mooney, M., Porat‐Dahlerbruch, J., Cahir, K., Ellen, M., Spinewine, A., Taylor, N., Laritz, R., Bourbeau-Allard, È., & Grimshaw, J. (2025) · Implementation Science · doi:10.1186/s13012-025-01436-5 · Cited by 39 · see citing works
Self-citationGoodson, C. M., Friedman, L. A., Mantheiy, E. C., Heckle, K., Lavezza, A., Toonstra, A., Parker, A. M., Seltzer, J., Velaetis, M., Glover, M., et al. (2018) · Journal of Intensive Care Medicine · doi:10.1177/0885066618807120 · Cited by 37 · see citing works
The use of music in aged care facilities: A mixed-methods study
Garrido, S., Dunne, L., Perz, J., Chang, E. M. L., & Stevens, C. (2018) · Journal of Health Psychology · doi:10.1177/1359105318758861 · Cited by 36 · see citing works
Thürlimann, E., Verweij, L., & Naef, R. (2022) · Journal of Family Nursing · doi:10.1177/10748407221099655 · Cited by 36 · see citing works
Geng, E., Mody, A., & Powell, B. J. (2023) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-051920-124515 · Cited by 36 · see citing works
Tan, X. J., Cheng, G., & Ling, M. H. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100521 · Cited by 39 · see citing works
Can feedback approaches reduce unwarranted clinical variation? A systematic rapid evidence synthesis
Harrison, R., Hinchcliff, R., Manias, E., Mears, S., Heslop, D. J., Walton, V., & Kwedza, R. (2020) · BMC Health Services Research · doi:10.1186/s12913-019-4860-0 · Cited by 34 · see citing works
Allen, C. G., Barbero, C., Shantharam, S. S., & Moeti, R. (2018) · Journal of Public Health Management and Practice · doi:10.1097/phh.0000000000000846 · Cited by 33 · see citing works
Deinse, T. B. V., Bunger, A. C., Burgin, S., Wilson, A. B., & Cuddeback, G. S. (2019) · Health & Justice · doi:10.1186/s40352-019-0098-5 · Cited by 32 · see citing works
Reid, H. A. B., Smith, R., Williamson, W., Baldock, J., Caterson, J., Kluzek, S., Jones, N., & Copeland, R. J. (2022) · BMC Public Health · doi:10.1186/s12889-022-14178-6 · Cited by 31 · see citing works
Midwives' use of best available evidence in practice: An integrative review
Leo, A. D., Bayes, S. J., Geraghty, S., & Butt, J. (2019) · Journal of Clinical Nursing · doi:10.1111/jocn.15027 · Cited by 31 · see citing works
Romney, W. M., Bellows, D. M., Tavernite, J. P., Salbach, N. M., & Deutsch, J. E. (2021) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2021.01.076 · Cited by 31 · see citing works
Jani, Y. H., Williams, I., & Krishna, M. T. (2019) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.14190 · Cited by 30 · see citing works
Weller, C. D., Richards, C., Turnour, L., & Victoria Team (2020) · International Wound Journal · doi:10.1111/iwj.13334 · Cited by 30 · see citing works
Rogal, S. S., Yakovchenko, V., Morgan, T. R. M., Bajaj, J. S., Gonzalez, R. I., Park, A., Beste, L. A., Miech, E. J., Lamorte, C., Neely, B., et al. (2020) · Implementation Science · doi:10.1186/s13012-020-01050-7 · Cited by 30 · see citing works
Rees, G. M., Bard, A., & Reyher, K. K. (2021) · Antibiotics · doi:10.3390/antibiotics10030253 · Cited by 30 · see citing works
Evidence-Based Communication Assessment and Intervention
Sigafoos, J., & Schlosser, R. W. (2018) · Oxford University Press eBooks · doi:10.1093/obo/9780199756810-0206 · Cited by 30 · see citing works
Bunger, A. C., Chuang, E., Girth, A. M., Lancaster, K. E., Gadel, F., Himmeger, M., Saldana, L., Powell, B. J., & Aarons, G. A. (2020) · Implementation Science · doi:10.1186/s13012-020-01016-9 · Cited by 29 · see citing works
Shrubsole, K., Worrall, L. E., & Power, E. (2018) · Aphasiology · doi:10.1080/02687038.2018.1510112 · Cited by 26 · see citing works
Development and Results of an Implementation Plan for High-Intensity Gait Training
Moore, J. L., Bø, E., Erichsen, A., Rosseland, I., Halvorsen, J., Bratlie, H., Hornby, T. G., & Nordvik, J. E. (2021) · Neurology Report · doi:10.1097/npt.0000000000000364 · Cited by 25 · see citing works
Baumann, H., Heuel, L., Bischoff, L. L., & Wollesen, B. (2023) · Sensors · doi:10.3390/s23042322 · Cited by 25 · see citing works
Soril, L. J. J., NOSEWORTHY, T. W., Dowsett, L. E., Memedovich, K. A., Holitzki, H., Lorenzetti, D., Stelfox, H. T., Zygun, D. A., & Clement, F. M. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019912 · Cited by 25 · see citing works
Soril, L. J. J., Seixas, B. V., Mitton, C., Bryan, S., & Clement, F. M. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3459-1 · Cited by 25 · see citing works
Holm, A., Karlsson, V., & Dreyer, P. (2021) · International Journal of Qualitative Studies on Health and Well-Being · doi:10.1080/17482631.2021.1971598 · Cited by 24 · see citing works
Watson, J. S., Jordan, R. E., Adab, P., Vlaev, I., Enocson, A., & Greenfield, S. (2022) · BMJ Open · doi:10.1136/bmjopen-2020-046875 · Cited by 24 · see citing works
Optimizing Implementation in Cancer Control (OPTICC): protocol for an implementation science center
Lewis, C. C., Hannon, P. A., Klasnja, P., Baldwin, L., Hawkes, R. J., Blackmer, J., & Johnson, A. N. (2021) · Implementation Science Communications · doi:10.1186/s43058-021-00117-w · Cited by 23 · see citing works
Ayivi‐Vinz, G., Kanyinga, F. B., Bergeron, L., Décary, S., Adisso, É. L., Zomahoun, H. T. V., Daniel, S. J., Tremblay, M., Plourde, K. V., Guay-Bélanger, S., et al. (2022) · JMIR Medical Education · doi:10.2196/36948 · Cited by 23 · see citing works
Kirk, J. W., Nilsén, P., Andersen, O., Powell, B. J., Tjørnhøj‐Thomsen, T., Bandholm, T., & Pedersen, M. M. (2022) · BMC Health Services Research · doi:10.1186/s12913-021-07395-z · Cited by 22 · see citing works
Shrubsole, K., Copland, D. A., Hill, A. J., Khan, A., Lawrie, M., O’Connor, D. A., Pattie, M., Rodriguez, A. D., Ward, E. C., Worrall, L. E., et al. (2022) · Aphasiology · doi:10.1080/02687038.2022.2095608 · Cited by 22 · see citing works
Wirtz, M. A., Bitzer, E. M., Albert, U., Ansmann, L., Bögel, M., Ernstmann, N., Hollederer, A., Hower, K. I., Nowak, M., & Vollmar, H. C. (2019) · Das Gesundheitswesen · doi:10.1055/a-0862-0588 · Cited by 22 · see citing works
Duke, S., Campling, N. C., May, C. R., Lund, S., Lunt, N. T., Bartlett, G., Harris, L., Flannery, E., Connolly, M., Booth, P., et al. (2020) · BMC Palliative Care · doi:10.1186/s12904-020-00647-5 · Cited by 22 · see citing works
Knoerl, R., Mazzola, E., Hong, F., Salehi, E., McCleary, N. J., Ligibel, J. A., Reyes, K., & Berry, D. L. (2021) · BMC Cancer · doi:10.1186/s12885-021-07965-8 · Cited by 22 · see citing works
Jung, O., Graetz, I., Dorner, S. C., & Hayden, E. M. (2022) · Medical Care Research and Review · doi:10.1177/10775587221108750 · Cited by 21 · see citing works
Identifying Barriers to Nurse-Facilitated Patient Mobility in the Intensive Care Unit
Young, D. L., Seltzer, J., Glover, M., Outten, C., Lavezza, A., Mantheiy, E. C., Parker, A. M., & Needham, D. M. (2018) · American Journal of Critical Care · doi:10.4037/ajcc2018368 · Cited by 21 · see citing works
Cahyanti, R. D., Widyawati, W., & Hakimi, M. (2021) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-021-04212-7 · Cited by 21 · see citing works
Holm, A., Karlsson, V., Nikolajsen, L., & Dreyer, P. (2021) · International Journal of Nursing Studies Advances · doi:10.1016/j.ijnsa.2021.100025 · Cited by 20 · see citing works
Nguyen, M. X. B., Chu, A. V., Powell, B. J., Tran, H. V., Nguyen, L. H., Dao, A. T. M., Pham, M. D., Vo, S. H., Bui, N. H., Dowdy, D. W., et al. (2020) · Implementation Science · doi:10.1186/s13012-020-01020-z · Cited by 19 · see citing works
Rouleau, G., Pelletier, J., Côté, J., Gagnon, M., Martel‐Laferrière, V., Lévesque, R., SimforHealth, & Fontaine, G. (2020) · Journal of Medical Internet Research · doi:10.2196/18225 · Cited by 19 · see citing works
Using self-determination theory to predict self-management and HRQoL in moderate-to-severe COPD
Knox, L., Norris, G., Lewis, K. E., & Rahman, R. J. (2021) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2021.1938073 · Cited by 19 · see citing works
Expanding the conversation: A Person-centred Communication Enhancement Model
O’Rourke, D. J., Lobchuk, M., Thompson, G. N., & Lengyel, C. O. (2022) · Dementia · doi:10.1177/14713012221080252 · Cited by 19 · see citing works
Nyman, M. H., Forsman, H., Wallin, L., Ostaszkiewicz, J., Hommel, A., & Eldh, A. C. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12879 · Cited by 19 · see citing works
Implementing clinical guidelines
Mayer, J., Kipps, C., & Cock, H. R. (2019) · Practical Neurology · doi:10.1136/practneurol-2017-001814 · Cited by 18 · see citing works
Raj, S. X., Brunelli, C., Klepstad, P., & Kaasa, S. (2017) · Scandinavian Journal of Pain · doi:10.1016/j.sjpain.2017.07.016 · Cited by 18 · see citing works
Cohen, R., Gesser‐Edelsburg, A., Singhal, A., Benenson, S., & Moses, A. E. (2022) · PLoS ONE · doi:10.1371/journal.pone.0269124 · Cited by 17 · see citing works
Quinn, E. D., Kurin, K., Atkins, K. L., & Cook, A. (2023) · Language Speech and Hearing Services in Schools · doi:10.1044/2023_lshss-22-00186 · Cited by 17 · see citing works
Schonnop, R., Dainty, K. N., McLeod, S. L., Melady, D., & Lee, J. S. (2022) · Canadian Journal of Emergency Medicine · doi:10.1007/s43678-022-00371-4 · Cited by 17 · see citing works
Lucas, P. J., Ingram, J. C., Redmond, N. M., Cabral, C., Turnbull, S., & Hay, A. D. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0455-9 · Cited by 17 · see citing works
Differences in implementation of family focused practice in hospitals: a cross-sectional study
Skogøy, B. E., Maybery, D. J., Ruud, T., Sørgaard, K. W., Peck, G. C., Kufås, E., Stavnes, K. A., Thorsen, E., & Ogden, T. (2018) · International Journal of Mental Health Systems · doi:10.1186/s13033-018-0256-5 · Cited by 17 · see citing works
Yazdizadeh, B., Walker, R., Skouteris, H., Olander, E. K., & Hill, B. (2020) · Health Promotion International · doi:10.1093/heapro/daaa039 · Cited by 16 · see citing works
Rodrigues, A., Nichol, B., Wilson, R., Charlton, C., Gibson, B., Finch, T. L., Haighton, C., Maniatopoulos, G., Giles, E. L., Harrison, D., et al. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-084208 · Cited by 16 · see citing works
Vaisson, G., Witteman, H. O., Dansokho, S. C., Saragosa, M., Bouck, Z., Bravo, C. A., Desveaux, L., Llovet, D., Presseau, J., Taljaard, M., et al. (2019) · Current Oncology · doi:10.3747/co.26.4829 · Cited by 16 · see citing works
Self-citationBélanger‐Gravel, A., Janezic, I., Desroches, S., Paquette, M.-C., Therrien, F., Barnett, T. A., Lavoie, K., & Gauvin, L. I. (2023) · BMC Health Services Research · doi:10.1186/s12913-023-09455-y · Cited by 16 · see citing works
Kolehmainen, N., Marshall, J., Hislop, J., Fayed, N., Kay, D., Ternent, L., & Pennington, L. (2019) · Child Care Health and Development · doi:10.1111/cch.12723 · Cited by 15 · see citing works
Self-citationRiordan, F., Curran, G. M., Pallin, N., Albers, B., Presseau, J., Morrissey, E., Curran, G. M., Lewis, C. C., Powell, B. J., Presseau, J., et al. (2022) · HRB Open Research · doi:10.12688/hrbopenres.13507.1 · Cited by 15 · see citing works
Mizrahi, D., Goldstein, D., Kiernan, M. C., Robinson, L., Pitiyarachchi, O., McCullough, S., Mendoza-Jones, P., Grimison, P., Boyle, F. M., & Park, S. B. (2022) · Supportive Care in Cancer · doi:10.1007/s00520-022-07024-3 · Cited by 15 · see citing works
McClatchey, K., Marsh, V., Steed, L., Holmes, S., Taylor, S. J., Wiener-Ogilvie, S., Neal, J., Last, R., Saxon, A., Pinnock, H., et al. (2022) · Trials · doi:10.1186/s13063-022-06147-6 · Cited by 14 · see citing works
Pereira, C. F., de Vargas, D., & Beeber, L. S. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1124295 · Cited by 14 · see citing works
Kanyinga, F. B., Gogovor, A., Dofara, S., Gadio, S., Tremblay, M., Daniel, S. J., Rivest, L.-P., & Légaré, F. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04597-3 · Cited by 13 · see citing works
Lockwood, C. S. (2017) · Journal of Korean Academy of Nursing Administration · doi:10.11111/jkana.2017.23.2.111 · Cited by 14 · see citing works
Purvis, T. L., Middleton, S., Craig, L., Kilkenny, M. F., Dale, S., Hill, K. M., D’Este, C. A., & Cadilhac, D. A.-M. (2019) · Implementation Science · doi:10.1186/s13012-019-0934-y · Cited by 14 · see citing works
Gahan, L. B., Gaffy, E., Dow, B., & Brijnath, B. (2019) · Journal of Elder Abuse & Neglect · doi:10.1080/08946566.2019.1682098 · Cited by 14 · see citing works
Sprague, C., Woollett, N., & Hatcher, A. M. (2020) · Global Public Health · doi:10.1080/17441692.2020.1780290 · Cited by 13 · see citing works
Knox, L., Gemine, R. E., Dunning, M., & Lewis, K. E. (2021) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2020-000800 · Cited by 13 · see citing works
Jordan, S. E., Banner, T., Gabe‐Walters, M., Mikhail, J. M., Round, J. A., Snelgrove, S. R., Storey, M., Wilson, D., & Hughes, D. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-023377 · Cited by 13 · see citing works
O’Neil, J., van Ierssel, J. J., & Sveistrup, H. (2019) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x19845466 · Cited by 12 · see citing works
Curtis, K. A., Elphick, T., Eyles, M., & Ruperto, K. (2020) · Implementation Science Communications · doi:10.1186/s43058-020-00045-1 · Cited by 12 · see citing works
Garrido, S., Markwell, H., Andreallo, F., & Hatcher, D. (2021) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s293764 · Cited by 12 · see citing works
- Authors
- Colquhoun, H. L., Squires, J. E., Kolehmainen, N., Fraser, C., & Grimshaw, J. M.
- Affiliations
- University of Toronto (CA)
- University of Ottawa (CA)
- Newcastle University (GB)
- University of Aberdeen (GB)
- Corresponding author
- Colquhoun, Heather L.
- Venue
- Implementation Science
- Published
- 2017-03-04
- Volume
- 12
- Issue
- 1
- Pages
- 30-30
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC5336662
- OpenAlex
- W2591591866
- Semantic Scholar
- 0c16198cd28cc786c55f201be8eefdb07af75558
- Cited by
- 283
- References
- 50
- Citations by year
- 2026: 11 · 2025: 25 · 2024: 40 · 2023: 35 · 2022: 41 · 2021: 40 · 2020: 27 · 2019: 28 · 2018: 25 · 2017: 10
- FWCI
- 4.34
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Implementation Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- TSpace (University of Toronto) (submittedVersion, cc-by)
- uO Research (University of Ottawa) (submittedVersion, public-domain)
- Newcastle University ePrints (Newcastle Univesity) (cc-by)
- Funders
- Canadian Institutes of Health Research
- Medical Research Council
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Psychology › Applied Psychology › Behavioral Health and Interventions
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Keywords
- intervention design, behaviour change interventions, barriers to change, theory-based intervention design, user engagement, organizational change, system-level interventions, design guidelines
- MeSH terms
- Health Personnel, Humans, Inservice Training, Health Knowledge, Attitudes, Practice, Professional Practice, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
