The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index; 4 of the 58 sources its claims cite were checked for support.
No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention
Open accessAbstract
BACKGROUND: Audit and feedback interventions in healthcare have been found to be effective, but there has been little progress with respect to understanding their mechanisms of action or identifying their key 'active ingredients.' DISCUSSION: Given the increasing use of audit and feedback to improve quality of care, it is imperative to focus further research on understanding how and when it works best. In this paper, we argue that continuing the 'business as usual' approach to evaluating two-arm trials of audit and feedback interventions against usual care for common problems and settings is unlikely to contribute new generalizable findings. Future audit and feedback trials should incorporate evidence- and theory-based best practices, and address known gaps in the literature. SUMMARY: We offer an agenda for high-priority research topics for implementation researchers that focuses on reviewing best practices for designing audit and feedback interventions to optimize effectiveness.
Overview
CoChat Summary
TL;DR
This paper argues that continuing “business as usual” two-arm trials of audit and feedback (A&F) versus usual care is unlikely to produce new generalizable findings, and instead proposes an agenda for optimizing A&F design. It summarizes consensus best practices and high-priority research topics from a 2012 international expert meeting, grounded in Cochrane review evidence of a median 4.3% absolute improvement.
Key findings
- The Cochrane review of 140 randomized A&F trials found a median 4.3% absolute improvement (interquartile range 0.5% to 16%) in provider compliance; one-quarter of interventions had relatively large positive effects, while another quarter had negative or null effects.
- Tentative best practices include valid and recent audit data about the individual/team’s own behavior, repeated audit cycles, multi-modal feedback, delivery from a trusted source, comparison data with relevant others, targets/goals, and a clear action plan when discrepancies are evident.
- A&F appears more effective when feedback is presented both verbally and in writing, comes from a supervisor or senior colleague, is repeated, and uses achievable benchmarks of care (top 10% of peers) rather than highlighting median peer performance; it is less effective when from a regulatory body.
- The authors recommend shifting future research from A&F versus usual care toward head-to-head comparisons of different A&F designs, fractional factorial trials, embedded process evaluations, and studies of contextual/recipient, intervention-design, and targeted-behavior factors.
Methods
- Design: a discussion/agenda paper summarizing outcomes from a two-day international expert meeting held in December 2012; participants were purposively sought based on contributions to A&F or related literature or as potential knowledge users, with a maximum of 30 attendees.
- Data: the most recent Cochrane review of A&F (140 randomized trials) and the collective experiences and research programs of meeting participants.
- Analysis: narrative synthesis and consensus discussion, producing Table 1 of tentative best practices and Table 2 of example high-priority research topics.
Limitations
- Stated in the paper: the recommendations reflect the limitations of the studies reviewed; the considerations only reflect opinions expressed at the meeting; with a maximum of 30 attendees, important perspectives may have been omitted; and there is no present universally accepted, comprehensive list of A&F intervention components.
- Evident from the text: the best practices are labeled “tentative,” and the paper does not conduct a formal systematic review, quantitative synthesis, or empirical validation of the proposed agenda.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 0 Not valid · 37 Not checked · 13 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Furthermore, most A&F interventions tested in RCTs are designed without explicitly building on previous research or extant theory [5,6].”
§ No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention > Background
What the sources say
- Not checked yetcites [5] Sales A, Smith J, Curran G, Kochevar L: Models, strategies, and tools. Theory in implementing evidence-based findings into health care practice. J Gen Intern Med. 2006, 21 (Suppl 2): 43-49.
- Partly supportedcites [6] Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, Carroll K, Chalifoux M, Eva KW: A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013, 8 (1): 66-10.1186/1748-5908-8-66. sourceExperimental — accuracy not yet measured
The Results and Discussion report that only 13 of 140 A&F trials (9%) explicitly used theory to develop the intervention, supporting the claim about extant theory. The review assesses reported theory use, not whether interventions built on previous research; it also cautions that authors may have used theory without reporting it. No page numbers are available in our copy. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Organizational and system-level factors that may affect recipients’ ability to act upon a discrepancy highlighted by feedback could also be considered, including competing priorities for the recipients, teams and organizations [34].”
§ No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention > Discussion > 1b. Applying relevant theory to improve design and increase contribution to the literature
What the sources say
- Partly supportedcites [34] Presseau J, Francis JJ, Campbell NC, Sniehotta FF: Goal conflict, goal facilitation, and health professionals’ provision of physical activity advice in primary care: An exploratory prospective study. Implement Sci. 2011, 6: 73-10.1186/1748-5908-6-73. sourceExperimental — accuracy not yet measured
Our copy supports only the 'competing priorities for the recipients' element: the Background states that clinical practice involves numerous goal-directed behaviours (GDBs) 'each competing for limited resources in patient consultations, in particular time-related resources', and the Discussion (Predicting health professional behaviour...) reports that the more participants perceived competing GDBs as taking time away from giving PA advice, the fewer patients received it. However, this is framed as the individual health professional's perception of intergoal conflict, not as 'organizational and system-level' factors, and nothing in our copy addresses competing priorities at the level of teams or organisations, nor feedback or acting on a highlighted discrepancy. The broader framing of the citing sentence therefore exceeds what Presseau et al. (2011) state. (LLM, conf=0.65)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Specifically, the theory of planned behaviour emphasizes the potential for social pressure to influence intentions to change professional practice [23]”
§ No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention > Discussion > 1a.i. A&F components
What the sources say
- Partly supportedcites [23] Ajzen I: The theory of planned behavior. Organ Behav Hum Decis Process. 1991, 50 (2): 179-211. 10.1016/0749-5978(91)90020-T. sourceExperimental — accuracy not yet measured
The section 'Predicting Intentions: Attitudes, Subjective Norms, and Perceived Behavioral Control' states that subjective norm is 'the perceived social pressure to perform or not to perform the behavior' and that, together with attitude and perceived behavioral control, it determines intention — so the social-pressure-to-intention part of the claim is directly supported (also the Abstract, which lists subjective norms as an intention predictor). What our copy does not support is the qualifier 'to change professional practice': the applications and studies it reports are leisure behavior, condom use, problem drinking, weight loss, and unethical acts among college students (cheating, shoplifting, lying), and its Conclusions frame the theory as a general framework for understanding or changing behavior in specified contexts, with no reference to professional or clinical practice. The citing sentence therefore extends this general theory paper to a professional-practice domain it does not address. (LLM, conf=0.78)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
In the paper
“Finally, the A&F review suggested that repeated feedback cycles led to greater improvements in performance than once-only feedback.”
§ No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention > Discussion > 1a.i. A&F components
What the sources say
- Not checked yetcites [3] Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD: Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012, 6: CD000259
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The challenge of identifying factors that differentiate more and less successful A&F interventions is exacerbated by poor reporting of both intervention components and contextual factors in the literature [4].”
§ No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention > Background
What the sources say
- Not checked yetcites [4] Glasziou P, Meats E, Heneghan C, Shepperd S: What is missing from descriptions of treatment in trials and reviews?. BMJ. 2008, 336 (7659): 1472-1474. 10.1136/bmj.39590.732037.47. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
27 more claims
Create a free account to see all 54 claims and to verify the 54 references still open.
- It may state two separate claims as one. 10
- Probably not a substantial claim. 14
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In fact, active research agendas have focused on determining the essential components of many of these “discrete” implementation strategies, such as audit and feedback [44], learning collaboratives [45], and supervision [46].”
From A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project (2015) · cites this paper as [44]
In the citing paper
“There are over 140 randomized trials of audit and feedback, but Ivers et al. (83) conclude that there is much to learn about when it will work best and why, and how to design reliable and effective audit and feedback strategies across different settings and providers.”
From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [83]
Citations
Cites
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
Building a practically useful theory of goal setting and task motivation: A 35-year odyssey.
Locke, E. A., & Latham, G. P. (2002) · American Psychologist · doi:10.1037/0003-066x.57.9.705 · Cited by 7,766 · cited 5×
“Alternatively, co-interventions that increase recipients’ skills in quality improvement may be useful because skill development and resultant selfefficacy may increase the likelihood that feedback will lead to improved performance [11,36].”
BackgroundMethodInfluentialKluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882 · cited 4×
“A&F has been used as a method for changing behaviour in educational [8], organizational [9-11], cognitive [12], and health psychology [13,14]; research in these areas has produced hypotheses about mechanisms of action.”
BackgroundMethodInfluentialAudit and feedback: effects on professional practice and healthcare outcomes
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., O'Brien, M. A., Johansen, M., Grimshaw, J., & Oxman, A. D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000259.pub3 · Cited by 5,485
Self-citationCane, J., O’Connor, D., & Michie, S. (2012) · Implementation Science · doi:10.1186/1748-5908-7-37 · Cited by 4,888 · cited 1×
“While the optimal design of A&F will vary to some extent for each purpose or setting, the strategies chosen for each component (Table 1) should be justified both by empirical evidence and theory [29].”
BackgroundSelf-citationShute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631
Gluesing, J. C. (2012) · Leadership in Science and Technology: A Reference Handbook · doi:10.4135/9781412994231.n14 · Cited by 3,210
Carver, C. S., & Scheier, M. F. (1982) · Psychological Bulletin · doi:10.1037//0033-2909.92.1.111 · Cited by 2,650 · cited 4×
“A&F has been used as a method for changing behaviour in educational [8], organizational [9-11], cognitive [12], and health psychology [13,14]; research in these areas has produced hypotheses about mechanisms of action.”
BackgroundMethodInfluentialDavis, D. A., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., & Perrier, L. (2006) · JAMA · doi:10.1001/jama.296.9.1094 · Cited by 2,375 · cited 1×
“A&F is an improvement over self-assessment [1] or self-monitoring [2] as it can provide objective data regarding discrepancies between current practice and target performance, as well as comparisons of performance to other health professionals.”
Background
Research works printed but not linked: 2
- Doran GT: There’s a S.M.A.R.T. way to write management’s goals and objectives. AMA Forum. Management review. 1981, 70 (11): 35-36.
- Ivers N, Barnsley J, Upshur R, Shah B, Tu K, Grimshaw J, Zwarenstein M: My job is one patient at a time’: perceived discordance between population-level quality targets and patient-centered care inhibits quality improvement. Canadian Family Physician. 2013, in press
Cited by
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015) · Implementation Science · doi:10.1186/s13012-015-0209-1 · Cited by 4,963
“In fact, active research agendas have focused on determining the essential components of many of these “discrete” implementation strategies, such as audit and feedback [44], learning collaboratives [45], and supervision [46].”
BackgroundEnhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda
Powell, B. J., Fernandez, M. E., Williams, N. J., Aarons, G. A., Beidas, R. S., Lewis, C. C., McHugh, S. M., & Weiner, B. J. (2019) · Frontiers in Public Health · doi:10.3389/fpubh.2019.00003 · Cited by 775
“(83) conclude that there is much to learn about when it will work best and why, and how to design reliable and effective audit and feedback strategies across different settings and providers.”
BackgroundPractice Feedback Interventions: 15 Suggestions for Optimizing Effectiveness
Brehaut, J. C., Colquhoun, H. L., Eva, K. W., Carroll, K., Sales, A., Michie, S., Ivers, N., & Grimshaw, J. M. (2016) · Annals of Internal Medicine · doi:10.7326/m15-2248 · Cited by 491
Self-citationLevers for addressing medical underuse and overuse: achieving high-value health care
Elshaug, A. G., Rosenthal, M. B., Lavis, J. N., Brownlee, S., Schmidt, H., Nagpal, S., Littlejohns, P., Srivastava, D., Tunis, S., & Saini, V. (2017) · The Lancet · doi:10.1016/s0140-6736(16)32586-7 · Cited by 440
Brown, B., Gude, W. T., Blakeman, T., van der Veer, S. N., Ivers, N., Francis, J. J., Lorencatto, F., Presseau, J., Peek, N., & Daker-White, G. (2019) · Implementation Science · doi:10.1186/s13012-019-0883-5 · Cited by 387
“Table 2 Popular theories to model clinical performance feedback in the literature [5]”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Ivers, N. M., Sales, A., Colquhoun, H., Michie, S., Foy, R., Francis, J. J., & Grimshaw, J. M.
- Affiliations
- University of Toronto (CA)
- University of Michigan (US)
- Ottawa Hospital (CA)
- University College London (GB)
- University of Leeds (GB)
- City, University of London (GB)
- University of Ottawa (CA)
- Corresponding author
- Ivers, Noah M
- ORCID
- Venue
- Implementation Science
- Published
- 2014-01-17
- Volume
- 9
- Issue
- 1
- Pages
- 14-14
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC3896824
- OpenAlex
- W2097831015
- Semantic Scholar
- bbd8c6056f8e825734b5543e95f88722fa0988e6
- Cited by
- 400
- References
- 52
- Citations by year
- 2026: 14 · 2025: 19 · 2024: 39 · 2023: 36 · 2022: 54 · 2021: 54 · 2020: 35 · 2019: 32 · 2018: 33 · 2017: 33 · 2016: 21 · 2015: 19 · 2014: 10
- FWCI
- 49.93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- City Research Online (City University London) (acceptedVersion)
- Deep Blue (University of Michigan) (submittedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- uO Research (University of Ottawa) (submittedVersion, public-domain)
- TSpace (University of Toronto) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- University of Toronto
- Canadian Institutes of Health Research
- Department of Family and Community Medicine, University of Toronto
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- audit and feedback, intervention design, mechanism of action, healthcare quality improvement, two-arm trials, usual care comparison, research agenda, implementation research
- MeSH terms
- Attitude of Health Personnel, Behavior, Feedback, Health Services Research, Humans, Medical Audit, Quality of Health Care, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. https://app.cochat.ai/papers/10.1186/1748-5908-9-14
- The paper
- Ivers, N. M., Sales, A., Colquhoun, H., Michie, S., Foy, R., Francis, J. J., & Grimshaw, J. M. (2014). No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implementation Science. https://doi.org/10.1186/1748-5908-9-14
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.