Tailored interventions to address determinants of practice

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Seven included studies are ongoing and seven are awaiting assessment.

“Seven are ongoing studies (Ongoing studies) and seven are awaiting assessment (Studies awaiting classification).”

May combine two claims.

The remaining 17 studies were not eligible for meta-regression because they either did not assess a suitable binary outcome or reported no data at baseline.

“The remaining 17 studies were not eligible for meta‐regression because they either did not assess a suitable binary outcome, or they reported no data at baseline.”

May combine two claims.

In this update, we identified an additional nine studies, seven of which had been published since the previous version of the review.

“In this update of our review of randomised trials of tailored implementation interventions, we identified an additional nine studies, seven of which had been published since the previous version of the review (Baker 2010).”

• cites Baker 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Twenty-three of the included studies had been included in the previous version of the review, but three studies from that review were excluded because their interventions were not assessed as meeting the criteria for tailoring.

“Twenty‐three of the studies had been included in the previous version of the review, but we excluded three studies from that review because the interventions were not assessed on this occasion as meeting the criteria for tailoring (Davies 2002; Sehgal 2002; Verhoeven 2005).”

• cites Davies 2002Not checked yet

  Not checked in the free run.

• cites Sehgal 2002 sourceNot checked yet

  Not checked in the free run.

• cites Verhoeven 2005 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

None of the assessed study attributes (risk of bias, explicit use of theory, adjustment to local factors, number of domains addressed) were found to be associated with the reported effectiveness of tailored interventions.

“Separate models were fitted dependent on the intervention delivered to the control group (none or non‐tailored), but none were found to be associated with the reported effectiveness of the tailored interventions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

We included 15 of 32 studies in the meta-regression analysis.

“We included 15 of 32 studies in the meta‐regression analysis (Table 2; Table 3).”

Classified as probably not a substantial claim.

We added nine studies to this review to bring the total number of included studies to 32 comparing an intervention tailored to address identified determinants of practice to no intervention or an intervention(s) not tailored to the determinants.

“We added nine studies to this review to bring the total number of included studies to 32 comparing an intervention tailored to address identified determinants of practice to no intervention or an intervention(s) not tailored to the determinants.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

We screened 9403 unique citations and reviewed the full text of 360.

“We screened 9403 unique citations (Figure 1) and reviewed the full text of 360.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Our overall finding is similar to that of the previous review.

“Despite the increase in the number of studies, however, our overall finding is similar to that of the previous review.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Clear reports of unintended effects of implementation interventions were not included in the studies.

“Clear reports of such effects were not included in the studies.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

We excluded 106 studies with reasons and 208 with quick reference to the full text.

“We excluded 106 with reasons provided in the Characteristics of excluded studies table and we excluded 208 with quick reference to the full text.”

May combine two claims.

The 17 studies not included in the meta-analysis had findings showing variable effectiveness consistent with the findings of the meta-regression, varying from no effect to moderate effect between studies and between outcomes within studies.

“The 17 studies not included in the meta‐analysis had findings showing variable effectiveness consistent with the findings of the meta‐regression.”

• Also stated in: § Summary of findings · plain

  “The effect of tailored interventions in these studies varied from no effect to moderate effect between studies and between outcomes within studies, a finding consistent with the meta‐regression”

May combine two claims.

Currently available studies have used different methods to identify determinants of practice and different approaches to selecting interventions to address the determinants.

“Currently available studies have used different methods to identify determinants of practice and different approaches to selecting interventions to address the determinants.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Where data were collected on the same group of patients throughout the trial (three studies), no problem with drop-outs was reported.

“Where data were collected on the same group of patients throughout the trial (three studies), no problem with drop‐outs was reported.”

The quote only partly states this claim.

In sensitivity analyses using higher ICC estimates, the pooled odds ratio for all 15 studies was 1.54 (95% CI 1.27 to 1.86, P < 0.001).

“For the main analysis with all 15 studies, the pooled odds ratio was 1.54 (95% CI 1.27 to 1.86, P value < 0.001).”

May combine two claims.

The majority of trials included in this review collected data before and after interventions on different patients, therefore drop-out at the patient level was not an issue for these trials.

“The majority of trials included in this review collected data before and after interventions on different patients, therefore drop‐out at the patient level was not an issue for these trials.”

Classified as probably not a substantial claim.

It is not possible to determine from the studies reviewed which methods of identifying determinants and tailoring interventions to account for them should be selected.

“Also, it is not possible to determine from the studies reviewed which methods of identifying determinants and tailoring interventions to account for them should be selected.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The previous version of the review included 26 trials.

“26 were included in the previous version (Baker 2010)”

• cites Baker 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

In some studies, adjustments were made at the level of the cluster (clinical team, hospital).

“Whether adjustment was made for local factors New In some studies, adjustments were made at the level of the cluster (clinical team, hospital), and these adjustments could potentially improve the intervention effect Add”

Probably not a substantial claim: may not pass S3 (load-bearing).

Three studies included in the previous version were excluded from this update because they were assessed as no longer meeting criteria for tailoring.

“We excluded three studies from this update that had been included in the previous version of the review because we assessed them as no longer meeting our criteria for tailoring.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The studies employed various interventions to improve professional practice.

“The studies employed various interventions to improve professional practice”

May combine two claims.

There were insufficient data in the included studies to assess healthcare outcomes, costs, and adverse events.

“Outcome: Healthcare outcomes Cluster‐RCTs Insufficient data in the included studies Outcome: Costs Cluster‐RCTs Insufficient data in the included studies Outcome: Adverse events Cluster‐RCTs Insufficient data in the included studies”

May combine two claims.

In the included studies, tailoring was undertaken in different ways and agreement on which methods should be used appeared to be absent.

“In the included studies, tailoring was undertaken in different ways and agreement on which methods should be used appeared to be absent.”

May combine two claims.

The methods of tailoring are not yet well developed and are not described in detail in published studies.

“the methods of tailoring are not yet well developed and are not described in detail in published studies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

It was possible to include 32 trials in this update.

“It was possible to include 32 trials in this update”

Probably not a substantial claim: may not pass S3 (load-bearing).

Nine new studies were identified and added in this update.

“New searches performed to December 2014 and nine new studies identified”

• Also stated in: § What's new · plain

  “We excluded three studies from the previous version of the review and added nine studies in this update, giving 32 studies in total.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The new search for studies and content updated did not change the conclusions.

“New search for studies and content updated (no change to conclusions)”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In this update, three studies were excluded from the previous version of the review and nine studies were added, giving 32 studies in total.

“We excluded three studies from the previous version of the review in this update, and added nine studies, giving 32 studies in total.”

May combine two claims.

Reviews in specific clinical fields discussed the possibility that tailored strategies might be more effective than strategies selected without taking account of determinants, but did not address the effect or costs of tailored interventions specifically.

“Although there are a number of reviews in specific clinical fields (Chaillet 2006; Kroenke 2000), which have discussed the possibility that tailored strategies might be more effective than strategies selected without taking account of determinants, these reviews did not address the effect or costs of tailored interventions specifically.”

• cites Chaillet 2006 sourceNot checked yet

  Not checked in the free run.

• cites Kroenke 2000 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Earlier versions of this review concluded that tailored interventions were more likely to improve professional practice than no intervention or dissemination of guidelines or educational materials alone.

“which concluded that tailored interventions were more likely to improve professional practice than no intervention or dissemination of guidelines or educational materials alone.”

Classified as probably not a substantial claim. May combine two claims.

Determinants of practice can impede attempts to change health professional behaviour and are believed to influence the success of improvement strategies.

“Attempts to change the behaviour of health professionals may be impeded by a variety of different barriers, obstacles, or factors (which we collectively refer to as determinants of practice).”

• Also stated in: § Background · hedged

  “determinants are believed to influence the success of improvement strategies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

There can be delays before the findings of research are widely adopted.

“there can be delays before the findings of research are widely adopted”

• cites Grol 2005Not checked yet

  Not checked in the free run.

• cites Oxman 1995Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

If determinants of practice are identified and strategies are implemented that have been chosen to address them, it is reasonable to expect performance to improve.

“it would appear reasonable to expect performance to improve”

Probably not a substantial claim: may not pass S3 (load-bearing).

The unexpected finding of a smaller effect compared with no intervention may be due to imprecision, chance, the small number of studies, and other unexplained factors.

“This unexpected finding may be due to imprecision, chance, the small number of studies, and other unexplained factors.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Two review authors independently classified the intensity of the methods used to identify determinants.

“Two review authors independently classified the intensity of the methods used to identify determinants using the following three categories”

May combine two claims.

It is not yet clear how best to tailor interventions and therefore not clear what the effect of an optimally tailored intervention would be; there is insufficient evidence on the most effective approaches to tailoring, including identification of determinants, prioritisation, and selection of interventions.

“It is not yet clear how best to tailor interventions and therefore not clear what the effect of an optimally tailored intervention would be.”

• Also stated in: § Plain language summary · plain

  “There is insufficient evidence on the most effective approaches to tailoring, including how determinants should be identified, how decisions should be made on which determinants are most important to address, and how interventions should be selected to account for the important determinants.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Meta-regression was used in the analyses to account for differences between control and intervention groups at study baselines.

“In our analyses, we used meta‐regression in order to be able to account for differences between control and intervention groups at study baselines.”

Probably not a substantial claim: may not pass S3 (load-bearing).

No language restrictions were applied in the searches or inclusion of studies.

“We applied no language restrictions in the searches or inclusion of studies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A sensitive search of major biomedical databases and trial registries was conducted.

“We conducted a sensitive search of major biomedical databases and trial registries (see Search methods for identification of studies).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Two review authors independently extracted the data from included studies.

“Two review authors independently extracted the data from included studies by using a revised version of the data extraction form used in the previous version of the review (Baker 2010).”

• cites Baker 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Two review authors assessed the risk of bias of the included studies using the Cochrane Handbook approach.

“Two review authors assessed the risk of bias of the included studies using the approach set out in Chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).”

• cites Higgins 2011Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Adverse effects of implementation interventions may have been unlikely, although unintended effects are possible.

“Since the studies were of interventions designed to implement appropriate clinical practice, adverse effects may have been unlikely, although it is possible that the implementation interventions could have unintended effects.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Two review authors independently assessed studies for inclusion.

“Two review authors independently assessed studies for inclusion.”

Classified as probably not a substantial claim. May combine two claims.

For the 15 studies included in the meta-regression, estimated odds ratios for each study (adjusted for clustering) were combined using meta-regression techniques, with baseline odds ratios included as a covariate to adjust for baseline differences between intervention and control groups.

“For the 15 studies included in the meta‐regression, we combined the estimated odds ratios for each study (adjusted for clustering) using meta‐regression techniques, whereby the baseline odds ratios were included as a covariate to adjust for any baseline differences between the intervention and control groups (Sutton 2000).”

• cites Sutton 2000Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Two independent review authors classified the extent to which the tailored intervention was adjusted to account for local factors.

“Two independent review authors classified the extent to which the tailored intervention was adjusted to account for local factors using the following two categories”

Probably not a substantial claim: may not pass S3 (load-bearing).

Bosch and colleagues undertook a qualitative analysis of 20 quality improvement studies reporting investigation of determinants.

“Bosch and colleagues undertook a qualitative analysis of 20 quality improvement studies reporting investigation of determinants (Bosch 2007).”

• cites Bosch 2007 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Heterogeneity for all meta-regression models was assessed using the I2 residual statistic, which represents the proportion of residual between-study variation due to heterogeneity rather than sampling variability.

“We assessed heterogeneity for all meta‐regression models using the I² residual statistic (Higgins 2003), which represents the proportion of residual between‐study variation due to heterogeneity, as opposed to sampling variability.”

• cites Higgins 2003 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

There is no equivalent of forest or funnel plots for meta-regression analyses, and reliance on only follow-up results to produce such plots would have provided misleading information.

“There is no equivalent of forest or funnel plots for meta‐regression analyses and reliance on only follow‐up results to produce such plots would have provided misleading information.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In many studies, the reasons for believing a particular strategy would address a particular determinant were not explained.

“but in many studies the reasons for believing a particular strategy would address a particular determinant were not explained.”

• cites Bosch 2007 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Sensitivity analyses in the meta-regression assumed a larger clustering effect than in the standard analyses by using higher ICC estimates taken from Campbell.

“In the meta‐regression analyses, we carried out sensitivity analyses assuming a larger clustering effect than had been accounted for in the standard analyses, by using higher ICC estimates (i.e. the reported upper quartile range values) taken from Campbell (Campbell 2005).”

• cites Campbell 2005 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Campbell et al extracted 220 ICCs from 21 data sets and reported the results separately for both primary and secondary care settings.

“Campbell et al extracted 220 ICCs from 21 data sets and reported the results separately for both primary and secondary care settings.”

• cites Campbell 2005 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

There may be additional evidence on the effectiveness of tailoring or on how it can be undertaken most effectively.

“Consequently, there may be additional evidence on the effectiveness of tailoring or on how it can be undertaken most effectively.”

May combine two claims.

Tailored implementation can be effective, but the effect is variable and tends to be small to moderate.

“Tailored implementation can be effective, but the effect is variable and tends to be small to moderate.”

• Also stated in: § Plain language summary · plain

  “The findings continue to indicate that tailored interventions can change professional practice, although they are not always effective and, when they are, the effect is small to moderate.”

• Also stated in: § Discussion · plain

  “Tailored implementation can be effective, but the effect is variable and tends to be small to moderate.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Strategies to address determinants may be selected in different ways.

“Once determinants have been identified and those to be addressed have been chosen, strategies to address them have to be selected, but this process may be undertaken in different ways (Wensing 2014).”

• cites Wensing 2014 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

It is possible that use of interventions such as audit with feedback or educational outreach would have improved professional practice whether or not tailoring had been undertaken.

“it is possible that use of such interventions (for example, audit with feedback, educational outreach) would have improved professional practice whether or not tailoring had been undertaken.”

Probably not a substantial claim: may not pass S3 (load-bearing).

If several methods are used together, a large number of determinants may be identified.

“if several methods are used together, a large number of determinants may be identified (Krause 2014a)”

• cites Krause 2014aNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Avorn 1992 had an unclear overall risk of bias.

“Risk of bias overall Unclear risk 5 criteria unclear risk”

The quote only partly states this claim.

For interventions tailored to address identified determinants of practice compared to no intervention, the certainty of evidence for implementation of recommended practice is moderate (3) for cluster-RCTs.

“Outcome: Implementation of recommended practice, e.g. clinical practice guideline recommendations 17 (7 studies included in meta‐regression) Cluster‐RCTs (4) Borderline risk of bias (‐0.5) Inconsistency between studies, and between outcomes within studies (‐0.5) No serious indirectness No serious imprecision None 3”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Using a complex search, including a sensitive RCT filter, in key electronic databases should have identified the majority of relevant, published trials.

“Using a complex search, including a sensitive RCT filter, in the key electronic databases should have identified the majority of relevant, published trials.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The overall risk of bias for Foy 2004 was unclear (4 criteria unclear risk, including randomisation and allocation concealment).

“Risk of bias overall Unclear risk 4 criteria unclear risk, including randomisation and allocation concealment”

• cites Foy 2004 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Avorn 1983 had an unclear overall risk of bias.

“Risk of bias overall Unclear risk 5 criteria unclear risk”

Probably not a substantial claim: may not pass S4 (based).

Adjustments at the level of the cluster could potentially improve the intervention effect.

“Whether adjustment was made for local factors New In some studies, adjustments were made at the level of the cluster (clinical team, hospital), and these adjustments could potentially improve the intervention effect Add”

Probably not a substantial claim: may not pass S3 (load-bearing).

Determinants may be classified in different ways.

“Determinants may be classified in different ways (Légaré 2009).”

• cites Légaré 2009Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Theories of human behaviour potentially help explain why a determinant affects practice and indicate which interventions are most likely to address the determinants.

“Use of a theory when developing an intervention In analysis Theories of human behaviour potentially help explain why a determinant affects practice and indicate which interventions are most likely to address the determinants Keep”

May combine two claims.

The meta-regression pooled a relatively wide variety of outcomes, but in all studies the outcomes related to processes of care and addressed the same question about effectiveness of tailored interventions.

“We pooled a relatively wide variety of outcomes in the meta‐regression analysis, although in all studies the study outcomes related to processes of care, and the studies all addressed the same question about the effectiveness of tailored interventions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The economic costs of tailoring may be higher than several other interventions because identification of determinants and tailoring involve additional steps.

“Since the identification of determinants and tailoring of strategies involve additional steps beyond the application of a particular strategy, such as education alone, the economic costs of tailoring may be higher than several other interventions.”

• AssumptionRests on an assumption: The identification of determinants and tailoring of strategies involve additional steps beyond the application of a particular strategy.

Probably not a substantial claim: may not pass S3 (load-bearing).

The overall risk of bias for Flottorp 2002 was unclear.

“Risk of bias overall Unclear risk Allocation concealment”

• cites Flottorp 2002 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The overall risk of bias for Callahan 1994 was unclear (5 criteria unclear risk).

“Risk of bias overall Unclear risk 5 criteria unclear risk”

• cites Callahan 1994 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The overall risk of bias for Engers 2005 was unclear (4 criteria unclear risk).

“Risk of bias overall Unclear risk 4 criteria unclear risk”

• cites Engers 2005 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

This risk is more likely to be a source of bias for reviews that include interrupted time series and controlled before-after studies, because randomised trials are more likely to be identified through electronic database searches.

“However, this is more likely to be a source of bias for reviews in which interrupted time series and controlled before‐after studies are included, since in comparison with these types of studies, randomised trials are more likely to be identified through electronic database searches.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The economic costs of tailoring may be lower than other interventions by enabling more expensive elements to be reserved for situations when they are likely to be effective.

“Conversely, they may be lower through enabling the more expensive elements of interventions to be reserved for situations when they are likely to be effective.”

May combine two claims.

In the meta-regression analysis, the outcomes included were either those reported as the primary outcome or, when this was not possible, the most clinically relevant measure, and therefore the introduction of bias is unlikely.

“In the meta‐regression analysis, the outcomes included were either those reported as the primary outcome or, when this was not possible, we selected the most clinically relevant measure and therefore the introduction of bias is unlikely.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Determinants may be investigated by various methods.

“Determinants may be investigated by various methods”

• cites Krause 2014aNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Looijmans 2010, contamination was at low risk of bias.

“Contamination Low risk Randomisation at nursing home level”

• cites Looijmans 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Looijmans 2010, allocation concealment was at unclear risk of bias.

“Allocation concealment (selection bias) Unclear risk Information not provided”

• cites Looijmans 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Looijmans 2010, incomplete outcome data was at low risk of bias.

“Incomplete outcome data (attrition bias) All outcomes Low risk 2 intervention nursing homes withdrew during the study, and 1 control home was excluded as no influenza vaccination was offered during the study period”

• cites Looijmans 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Leviton 1999, blinding of outcome assessment was at unclear risk of bias.

“Blinding of outcome assessment (detection bias) All outcomes Unclear risk Data collection done by medical abstractors but it is not clear if they were blinded”

• cites Leviton 1999 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Langham 2002, incomplete outcome data was at low risk of bias.

“Incomplete outcome data (attrition bias) All outcomes Low risk Follow‐up of 85% of patients achieved”

• cites Langham 2002Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Lakshminarayan 2010, random sequence generation was at unclear risk of bias.

“Random sequence generation (selection bias) Unclear risk A randomised block design was used, but the method for generating the randomisation sequence was not described”

• cites Lakshminarayan 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Leviton 1999, overall risk of bias was unclear.

“Risk of bias overall Unclear risk 1 high risk and 5 unclear risk criteria”

• cites Leviton 1999 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Looijmans 2010, blinding of outcome assessment was at low risk of bias.

“Blinding of outcome assessment (detection bias) All outcomes Low risk The vaccinating professional recorded the vaccination on a website”

• cites Looijmans 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Looijmans 2010, baseline outcomes were at low risk of bias.

“Baseline outcomes Low risk Baseline outcomes similar”

• cites Looijmans 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Karuza 1995, random sequence generation was at unclear risk of bias.

“Random sequence generation (selection bias) Unclear risk Details not reported”

• cites Karuza 1995 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Langham 2002, contamination was at unclear risk of bias.

“Contamination Unclear risk Practices were randomised”

• cites Langham 2002Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Langham 2002, selective reporting was at low risk of bias.

“Selective reporting (reporting bias) Low risk Appropriate outcomes reported”

• cites Langham 2002Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Lakshminarayan 2010, selective reporting was at low risk of bias.

“Selective reporting (reporting bias) Low risk Appropriate outcomes reported”

• cites Lakshminarayan 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Lakshminarayan 2010, baseline outcomes were at unclear risk of bias.

“Baseline outcomes Unclear risk The baseline outcomes were similar, except for onset to drug treatment within 180 minutes; not clear if this was adjusted for”

• cites Lakshminarayan 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Langham 2002, baseline outcomes were at low risk of bias.

“Baseline outcomes Low risk Baseline outcomes similar”

• cites Langham 2002Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Lakshminarayan 2010, incomplete outcome data was at low risk of bias.

“Incomplete outcome data (attrition bias) All outcomes Low risk No loss to follow‐up after randomisation”

• cites Lakshminarayan 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Karuza 1995, selective reporting was at low risk of bias.

“Selective reporting (reporting bias) Low risk Appropriate outcomes reported”

• cites Karuza 1995 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Looijmans 2010, selective reporting was at low risk of bias.

“Selective reporting (reporting bias) Low risk Appropriate outcomes reported”

• cites Looijmans 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Leviton 1999, allocation concealment was at high risk of bias.

“Allocation concealment (selection bias) High risk Allocation was not undertaken independently”

• cites Leviton 1999 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Karuza 1995, incomplete outcome data was at unclear risk of bias.

“Incomplete outcome data (attrition bias) All outcomes Unclear risk It is not clear why the mean number of charts reviewed is lower than the 45 planned for; some charts were not available at post intervention data collection”

• cites Karuza 1995 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Looijmans 2010, baseline characteristics were at low risk of bias.

“Baseline characteristics Low risk Baseline characteristics comparable”

• cites Looijmans 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Lakshminarayan 2010, overall risk of bias was unclear.

“Risk of bias overall Unclear risk 4 unclear risk criteria”

• cites Lakshminarayan 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Karuza 1995, blinding of outcome assessment was at unclear risk of bias.

“Blinding of outcome assessment (detection bias) All outcomes Unclear risk Data were collected from patient charts and it is not clear whether the data collectors were blinded”

• cites Karuza 1995 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Karuza 1995, allocation concealment was at unclear risk of bias.

“Allocation concealment (selection bias) Unclear risk Details not clear.”

• cites Karuza 1995 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Leviton 1999, selective reporting was at low risk of bias.

“Selective reporting (reporting bias) Low risk Appropriate outcomes reported”

• cites Leviton 1999 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Langham 2002, overall risk of bias was unclear.

“Risk of bias overall Unclear risk 1 high risk, 5 unclear risk criteria”

• cites Langham 2002Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Leviton 1999, incomplete outcome data was at low risk of bias.

“Incomplete outcome data (attrition bias) All outcomes Low risk Only 1 hospital withdrew after randomisation”

• cites Leviton 1999 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Lakshminarayan 2010, baseline characteristics were at low risk of bias.

“Baseline characteristics Low risk Patient characteristics were similar”

• cites Lakshminarayan 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Langham 2002, baseline characteristics were at unclear risk of bias.

“Baseline characteristics Unclear risk Baseline patient characteristics comparable other than age, BP, and smoking prevalence in 1 group, but not clear whether adjusted for”

• cites Langham 2002Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Leviton 1999, baseline outcomes were at low risk of bias.

“Baseline outcomes Low risk Baseline outcomes were similar”

• cites Leviton 1999 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Lakshminarayan 2010, contamination was at low risk of bias.

“Contamination Low risk Hospitals randomised”

• cites Lakshminarayan 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Lakshminarayan 2010, other bias was at low risk of bias.

“Other bias Low risk Low likelihood of contamination; baseline characteristics of patients similar”

• cites Lakshminarayan 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Leviton 1999, contamination was at low risk of bias.

“Contamination Low risk The hospital was the unit of randomisation”

• cites Leviton 1999 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Karuza 1995, contamination was at low risk of bias.

“Contamination Low risk Practice groups randomised”

• cites Karuza 1995 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Langham 2002, other bias was at unclear risk of bias.

“Other bias Unclear risk Baseline patients' characteristics broadly similar; practices were in a single locality and contamination may have been possible”

• cites Langham 2002Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Karuza 1995, baseline characteristics were at low risk of bias.

“Baseline characteristics Low risk No patient differences between study groups”

• cites Karuza 1995 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Murphy 2009, blinding of outcome assessment was at unclear risk of bias.

“Blinding of outcome assessment (detection bias) All outcomes Unclear risk Data were collected from patient records by a research nurse”

• cites Murphy 2009 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Ross‐Degnan 1996, contamination was at low risk of bias.

“Contamination Low risk Pharmacies stratified by geographical location prior to randomisation”

• cites Ross‐Degnan 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Murphy 2009, baseline outcomes were at low risk of bias.

“Baseline outcomes Low risk Baseline outcomes comparable”

• cites Murphy 2009 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Ross‐Degnan 1996, incomplete outcome data was at low risk of bias.

“Incomplete outcome data (attrition bias) All outcomes Low risk Data were collected from all pharmacies in the study”

• cites Ross‐Degnan 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Santoso 1996, selective reporting was at low risk of bias.

“Selective reporting (reporting bias) Low risk Appropriate outcomes reported”

• cites Santoso 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Ross‐Degnan 1996, other bias was at unclear risk of bias.

“Other bias Unclear risk There are no baseline performance data; the risk of contamination is unclear”

• cites Ross‐Degnan 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Schouten 2007, baseline characteristics were at low risk of bias.

“Baseline characteristics Low risk Characteristics of hospitals and professionals similar at baseline, although there was an uneven distribution of patients with heart failure, but there was adjustment”

• cites Schouten 2007Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Ross‐Degnan 1996, allocation concealment was at unclear risk of bias.

“Allocation concealment (selection bias) Unclear risk Unable to assess”

• cites Ross‐Degnan 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Matchar 2002, other bias was at unclear risk of bias.

“Other bias Unclear risk Contamination may have been possible, since geographically related practices were randomised to intervention and control groups. Patient characteristics were similar in both groups”

• cites Matchar 2002 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Matchar 2002, contamination was at low risk of bias.

“Contamination Low risk Groups of geographically related practices were randomised”

• cites Matchar 2002 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Looijmans 2010, overall risk of bias was unclear.

“Risk of bias overall Unclear risk 2 unclear risk criteria (1 allocation concealment)”

• cites Looijmans 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Santoso 1996, baseline characteristics were at unclear risk of bias.

“Baseline characteristics Unclear risk Baseline characteristics not reported”

• cites Santoso 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Ross‐Degnan 1996, baseline outcomes were at low risk of bias.

“Baseline outcomes Low risk Baseline outcomes similar”

• cites Ross‐Degnan 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Looijmans 2010, other bias was at low risk of bias.

“Other bias Low risk Comparable baseline characteristics in the study groups; contamination unlikely”

• cites Looijmans 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Matchar 2002, selective reporting was at low risk of bias.

“Selective reporting (reporting bias) Low risk Appropriate outcomes reported”

• cites Matchar 2002 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Schouten 2007, random sequence generation was at unclear risk of bias.

“Random sequence generation (selection bias) Unclear risk Coin toss was used to assign hospitals to intervention or control groups”

• cites Schouten 2007Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Santoso 1996, contamination was at low risk of bias.

“Contamination Low risk Randomisation at the level of the district”

• cites Santoso 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Murphy 2009, contamination was at low risk of bias.

“Contamination Low risk Randomisation at general practice level”

• cites Murphy 2009 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Matchar 2002, baseline outcomes were at low risk of bias.

“Baseline outcomes Low risk No differences at baseline”

• cites Matchar 2002 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Ross‐Degnan 1996, overall risk of bias was unclear.

“Risk of bias overall Unclear risk 4 unclear risk, 1 high risk criteria”

• cites Ross‐Degnan 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Murphy 2009, other bias was at low risk of bias.

“Other bias Low risk Baseline characteristics similar; low likelihood of contamination”

• cites Murphy 2009 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Santoso 1996, other bias was at unclear risk of bias.

“Other bias Unclear risk Contamination is unlikely; baseline characteristics of the health centres are not reported”

• cites Santoso 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Santoso 1996, baseline outcomes were at low risk of bias.

“Baseline outcomes Low risk Baseline outcomes similar”

• cites Santoso 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Matchar 2002, allocation concealment was at unclear risk of bias.

“Allocation concealment (selection bias) Unclear risk Process not reported”

• cites Matchar 2002 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Murphy 2009, incomplete outcome data was at low risk of bias.

“Incomplete outcome data (attrition bias) All outcomes Low risk”

• cites Murphy 2009 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Schouten 2007, baseline outcomes were at low risk of bias.

“Baseline outcomes Low risk There were baseline differences in some outcomes, but adjusted for”

• cites Schouten 2007Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Murphy 2009, baseline characteristics were at low risk of bias.

“Baseline characteristics Low risk Intervention and control groups comparable”

• cites Murphy 2009 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Santoso 1996, allocation concealment was at unclear risk of bias.

“Allocation concealment (selection bias) Unclear risk Details not provided”

• cites Santoso 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Ross‐Degnan 1996, selective reporting was at low risk of bias.

“Selective reporting (reporting bias) Low risk Appropriate outcomes reported”

• cites Ross‐Degnan 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Matchar 2002, random sequence generation was at low risk of bias.

“Random sequence generation (selection bias) Low risk A computer random number function was used”

• cites Matchar 2002 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Matchar 2002, baseline characteristics were at unclear risk of bias.

“Baseline characteristics Unclear risk There were no differences in patient characteristics between study groups. No data on clinicians”

• cites Matchar 2002 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Matchar 2002, blinding of outcome assessment was at unclear risk of bias.

“Blinding of outcome assessment (detection bias) All outcomes Unclear risk Whether or not the data abstractors were blinded is not reported”

• cites Matchar 2002 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Murphy 2009, overall risk of bias was unclear.

“Risk of bias overall Unclear risk 3 unclear risk criteria”

• cites Murphy 2009 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Ross‐Degnan 1996, baseline characteristics were at unclear risk of bias.

“Baseline characteristics Unclear risk Baseline characteristics not reported”

• cites Ross‐Degnan 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Santoso 1996, blinding of participants and personnel was at unclear risk of bias.

“Blinding of participants and personnel (performance bias) All outcomes Unclear risk It would not have been possible to blind staff of the study health centres to the intervention group”

• cites Santoso 1996 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In van Bruggen 2008, allocation concealment was at low risk of bias.

“Allocation concealment (selection bias) Low risk Allocation was undertaken by an independent researcher”

• cites van Bruggen 2008 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Scott 2013, other bias was at low risk of bias.

“Other bias Low risk Low risk of contamination”

• cites Scott 2013 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Simon 2005, other bias was at unclear risk of bias.

“Other bias Unclear risk The study took place in a single health maintenance organisation and contamination may be a risk. Baseline adherence was higher than national figures”

• cites Simon 2005 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Schouten 2007, blinding of outcome assessment was at unclear risk of bias.

“Blinding of outcome assessment (detection bias) All outcomes Unclear risk It is not clear whether the data collectors were blind to study groups”

• cites Schouten 2007Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In van Bruggen 2008, incomplete outcome data was at low risk of bias.

“Incomplete outcome data (attrition bias) All outcomes Low risk No practices withdrew”

• cites van Bruggen 2008 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Simon 2005, baseline characteristics were at low risk of bias.

“Baseline characteristics Low risk More physicians in the control group were male. Otherwise, patient and physician characteristics were similar between groups (adjusted for)”

• cites Simon 2005 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Schouten 2007, other bias was at low risk of bias.

“Other bias Low risk Contamination unlikely; baseline characteristics similar”

• cites Schouten 2007Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).