Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Randomised controlled trials of implementation strategies tell us whether (or not) an intervention results in changes in professional behaviour but little about the causal mechanisms that produce any change. Theory-based process evaluations collect data on theoretical constructs alongside randomised trials to explore possible causal mechanisms and effect modifiers. This is similar to measuring intermediate endpoints in clinical trials to further understand the biological basis of any observed effects (for example, measuring lipid profiles alongside trials of lipid lowering drugs where the primary endpoint could be reduction in vascular related deaths).This study protocol describes a theory-based process evaluation alongside the Ontario Printed Educational Message (OPEM) trial. We hypothesize that the OPEM interventions are most likely to operate through changes in physicians' behavioural intentions due to improved attitudes or subjective norms with little or no change in perceived behavioural control. We will test this hypothesis using a well-validated social cognition model, the theory of planned behaviour (TPB) that incorporates these constructs. METHODS/DESIGN: We will develop theory-based surveys using standard methods based upon the TPB for the second and third replications, and survey a subsample of Ontario family physicians from each arm of the trial two months before and six months after the dissemination of the index edition of informed, the evidence based newsletter used for the interventions. In the third replication, our study will converge with the "TRY-ME" protocol (a second study conducted alongside the OPEM trial), in which the content of educational messages was constructed using both standard methods and methods informed by psychological theory. We will modify Dillman's total design method to maximise response rates. Preliminary analyses will initially assess the internal reliability of the measures and use regression to explore the relationships between predictor and dependent variable (intention to advise diabetic patients to have annual retinopathy screening and to prescribe thiazide diuretics for first line treatment of uncomplicated hypertension). We will then compare groups using methods appropriate for comparing independent samples to determine whether there have been changes in the predicted constructs (attitudes, subjective norms, or intentions) across the study groups as hypothesised, and will assess the convergence between the process evaluation results and the main trial results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
The Theory of Planned Behavior: A Review of its Applications to Health-Related Behaviors
Godin, G., & Kok, G. (1996) · American Journal of Health Promotion · doi:10.4278/0890-1171-11.2.87 · Cited by 3,185
Self-citationEffectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., Vale, L., Whitty, P., Eccles, M., Matowe, L., Shirran, L., et al. (2004) · Health Technology Assessment · doi:10.3310/hta8060 · Cited by 3,044 · cited 1×
“However the current evidence base [4] indicates that the effects of interventions do appear to vary by condition, professional group, and context, presumably because the causal mechanisms of the interventions are modified in the presence of different barriers and enablers.”
Self-citationFerlie, E. B., & Shortell, S. M. (2001) · The Milbank Quarterly · doi:10.1111/1468-0009.00206 · Cited by 1,335 · cited 1×
“Ferlie and Shortell [7] have suggested four levels at which knowledge translation interventions might operate: the individual health professional; health care groups or teams; organisations providing health care; and the larger health care system or environment in which individual organizations are embedded.”
Cited by
Action, actor, context, target, time (AACTT): a framework for specifying behaviour
Presseau, J., McCleary, N., Lorencatto, F., Patey, A. M., Grimshaw, J. M., & Francis, J. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0951-x · Cited by 411
Self-citationDesigning and undertaking randomised implementation trials: guide for researchers
Wolfenden, L., Foy, R., Presseau, J., Grimshaw, J. M., Ivers, N. M., Powell, B. J., Taljaard, M., Wiggers, J., Sutherland, R., Nathan, N., et al. (2021) · BMJ · doi:10.1136/bmj.m3721 · Cited by 270
Self-citationGeorgeu, D., Colvin, C. J., Lewin, S., Fairall, L., Bachmann, M. O., Uebel, K., Zwarenstein, M., Draper, B., & Bateman, E. D. (2012) · Implementation Science · doi:10.1186/1748-5908-7-66 · Cited by 115
Self-citationBamford, C., Heaven, B., May, C., & Moynihan, P. (2012) · Implementation Science · doi:10.1186/1748-5908-7-106 · Cited by 100
Curran, J. A., Brehaut, J., Patey, A. M., Osmond, M., Stiell, I., & Grimshaw, J. M. (2013) · Implementation Science · doi:10.1186/1748-5908-8-25 · Cited by 107
Self-citation
Metadata
- Authors
- Grimshaw, J. M., Zwarenstein, M., Tetroe, J. M., Godin, G., Graham, I. D., Lemyre, L., Eccles, M. P., Johnston, M., Francis, J. J., Hux, J., O'Rourke, K., Légaré, F., & Presseau, J.
- Venue
- Implementation Science
- Published
- 2007-11-26
- Volume
- 2
- Issue
- 1
- Pages
- 38-38
- ISSN
- 1748-5908 (ISSN-L)
- OpenAlex
- W1984234647
- Cited by
- 86
- References
- 4
- FWCI
- 6.98
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada. https://app.cochat.ai/papers/10.1186/1748-5908-2-38
- The paper
- Grimshaw, J. M., Zwarenstein, M., Tetroe, J. M., Godin, G., Graham, I. D., Lemyre, L., Eccles, M. P., Johnston, M., Francis, J. J., Hux, J., O'Rourke, K., Légaré, F., & Presseau, J. (2007). Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada. Implementation Science. https://doi.org/10.1186/1748-5908-2-38
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.