Meta-Analysis: Audit and Feedback Features Impact Effectiveness on Care Quality
Open accessAbstract
BACKGROUND: Audit and feedback (A&F) has long been used to improve quality of care, albeit with variable results. This meta-analytic study tested whether Feedback Intervention Theory, a framework from industrial/organizational psychology, explains the observed variability in health care A&F research. DATA SOURCE: studies cited by Jamtvedt's 2006 Cochrane systematic review of A&F, followed by database searches using the Cochrane review's search strategy to identify more recent studies. INCLUSION CRITERIA: Cochrane review criteria, plus: presence of a treatment group receiving only A & F; a control group receiving no intervention; a quantitatively measurable outcome; minimum n of 10 per arm; sufficient statistics for effect size calculations. Moderators: presence of discouragement and praise; correct solution, attainment level, velocity, frequency, and normative information; feedback format (verbal, textual, graphic, public, computerized, group vs. individual); goal setting activity. PROCEDURE: meta-analytic procedures using the Hedges-Olkin method. RESULTS: Of 519 studies initially identified, 19 met all inclusion criteria. Studies were most often excluded due to the lack of a feedback-only arm. A&F has a modest, though significant positive effect on quality outcomes (d = 0.40, 95% confidence interval = +/-0.20); providing specific suggestions for improvement, written, and more frequent feedback strengthened this effect, whereas graphical and verbal feedback attenuated this effect. CONCLUSIONS: A&F effectiveness is improved when feedback is delivered with specific suggestions for improvement, in writing, and frequently. Other feedback characteristics could also potentially improve effectiveness; however, research with stricter experimental controls is needed to identify the specific feedback characteristics that maximize its effectiveness.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Feedback is more effective when accompanied by both explicit goals and an action plan.
“resulting in the finding that feedback is more effective when accompanied by both explicit goals and an action plan.”
From No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention (2014) · cites this paper as [17]
Hysong has produced a meta-analysis that documents how different features of audit and feedback impact its effectiveness.
“Hysong [68] has produced a meta-analysis that documents how different features of audit and feedback impact its effectiveness.”
From Implementation strategies: recommendations for specifying and reporting (2013) · cites this paper as [68]
Variations in audit and feedback matter.
“These variations matter (84), and there is a need for tests of audit and feedback and other discrete implementation strategies that include clearly described components that are theoretically and empirically derived, and well-operationalized.”
From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [84]
Hysong's re-analysis of the 2006 Cochrane review based on Feedback Intervention Theory showed greater effectiveness with increasing frequency of feedback.
“The results showed greater effectiveness with increasing frequency of the feedback,”
From Audit and feedback: effects on professional practice and healthcare outcomes (2012) · cites this paper as Hysong 2009
Hysong's re-analysis showed greater effectiveness with written rather than verbal or graphical delivery.
“with written rather than verbal or graphical delivery”
From Audit and feedback: effects on professional practice and healthcare outcomes (2012) · cites this paper as Hysong 2009
Hysong's re-analysis showed greater effectiveness with feedback that included information about the correct solution.
“and with feedback that included information about the correct solution.”
From Audit and feedback: effects on professional practice and healthcare outcomes (2012) · cites this paper as Hysong 2009
Our results regarding action-planning are consistent with the re-analysis of the previous version of the review informed by the Feedback Intervention Theory.
“Our results regarding action‐planning are also consistent with the re‐analysis of the previous version of the review informed by the Feedback Intervention Theory (Hysong 2009).”
From Audit and feedback: effects on professional practice and healthcare outcomes (2012) · cites this paper as Hysong 2009
The findings of this review regarding format and source were consistent with a re-analysis of a previous version of the review.
“The findings of this review regarding format and source were consistent with a re‐analysis of a previous version of the review (Hysong 2009)”
From Audit and feedback: effects on professional practice and healthcare outcomes (2012) · cites this paper as Hysong 2009
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1 retracted
O’Brien, M. A., Oxman, A. D., Davis, D., Haynes, R. B., Freemantle, N., & Harvey, E. (1998) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000260 · Cited by 123 · see citing works
RetractedKluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882 · see citing works
Audit and feedback: effects on professional practice and healthcare outcomes
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., O'Brien, M. A., Johansen, M., Grimshaw, J., & Oxman, A. D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000259.pub3 · Cited by 5,485 · see citing works
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., Vale, L., Whitty, P., Eccles, M., Matowe, L., Shirran, L., et al. (2004) · Health Technology Assessment · doi:10.3310/hta8060 · Cited by 3,044 · see citing works
Davis, D. A., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., & Perrier, L. (2006) · JAMA · doi:10.1001/jama.296.9.1094 · Cited by 2,375 · see citing works
(2001) · PubMed · Cited by 1,709 · see citing works
Audit and feedback: effects on professional practice and health care outcomes
Jamtvedt, G., Young, J. M., Kristoffersen, D. T., O'Brien, M. A., & Oxman, A. D. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000259.pub2 · Cited by 912 · cited 7× · see citing works
“Kluger and DeNisi proposed that 3 factors determine how effectively this attentional shift occurs: (1) characteristics of the feedback itself (these can be content or format related), (2) the nature of the task performed, and (3) situational and personality variables.”
BackgroundInfluentialEffectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J. M., Thomas, R. E., MacLennan, G., Fraser, C., Ramsay, C. R., Vale, L., Whitty, P., Eccles, M. P., Matowe, L., Shirran, L., et al. (2005) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462305290190 · Cited by 864 · see citing works
Does Pay-for-Performance Improve the Quality of Health Care?
Petersen, L. A., Woodard, L., Urech, T. H., Daw, C., & Chitwood, S. S. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-4-200608150-00006 · Cited by 698 · see citing works
Audit and feedback: Effects on professional practice and healthcare outcomes
Jamtvedt, G., Jm, Y., Kristoffersen, D., O'Brien, M. T., & Oxman, A. D. (2003) · Physiotherapy · doi:10.1016/s0031-9406(05)60176-9 · Cited by 582 · see citing works
Implementing clinical guidelines: Current evidence and future implications
Grimshaw, J., Eccles, M., & Tetroe, J. (2004) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340240506 · Cited by 482 · see citing works
Evidence-Based Implementation of Evidence-Based Medicine
Grol, R., & Grimshaw, J. (1999) · The Joint Commission Journal on Quality Improvement · doi:10.1016/s1070-3241(16)30464-3 · Cited by 421 · see citing works
Smith, M. C. (1991) · doi:10.1007/978-1-349-21542-3_4 · Cited by 400 · see citing works
Audit and feedback and clinical practice guideline adherence: Making feedback actionable
Hysong, S. J., Best, R. G., & Pugh, J. A. (2006) · Implementation Science · doi:10.1186/1748-5908-1-9 · Cited by 371 · see citing works
Self-citationEffects of group feedback, goal setting, and incentives on organizational productivity.
Pritchard, R. D., Jones, S. D., Roth, P. L., Stuebing, K. K., & Ekeberg, S. E. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.2.337 · Cited by 369 · see citing works
Overton, R. C. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.3.354 · Cited by 353 · see citing works
Effects of feedback of information on clinical practice: a review.
Mugford, M., Banfield, P., & O'Hanlon, M. (1991) · BMJ · doi:10.1136/bmj.303.6799.398 · Cited by 333 · see citing works
Kluger, A. N., & DeNisi, A. S. (1998) · Current Directions in Psychological Science · doi:10.1111/1467-8721.ep10772989 · Cited by 314 · see citing works
SIOP (2002) · PsycEXTRA Dataset · doi:10.1037/e553002012-001 · Cited by 277 · see citing works
Foy, R., Eccles, M., Jamtvedt, G., Young, J., Grimshaw, J., & Baker, R. (2005) · BMC Health Services Research · doi:10.1186/1472-6963-5-50 · Cited by 250 · see citing works
A Covariance Structure Analysis of Employees' Response to Performance Feedback.
Kinicki, A. J., Prussia, G. E., Wu, B. (., & McKee‐Ryan, F. M. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.6.1057 · Cited by 214 · see citing works
Electronic medical record use by office-based physicians and their practices: United States, 2006.
Hing, E., Burt, C. W., & Woodwell, D. A. (2007) · PubMed · Cited by 145 · see citing works
Motivational effects of knowledge of results: Knowledge or goal setting?
Locke, E. A. (1967) · Journal of Applied Psychology · doi:10.1037/h0024771 · Cited by 131 · see citing works
Evaluating Meta-analyses in the General Surgical Literature: A Critical Appraisal
Dixon, E., Hameed, M., Sutherland, F., Cook, D. J., & Doig, C. (2005) · Annals of Surgery · doi:10.1097/01.sla.0000154258.30305.df · Cited by 129 · see citing works
Bentz, C. J., Bayley, B., Bonin, K., Fleming, L., Hollis, J. F., Hunt, J. S., LeBlanc, B., McAfee, T. A., Payne, N., & Siemienczuk, J. (2007) · Nicotine & Tobacco Research · doi:10.1080/14622200701188828 · Cited by 126 · see citing works
(1998) · PsycEXTRA Dataset · doi:10.1037/e583352006-003 · Cited by 104 · see citing works
Not of One Mind: Mental Models of Clinical Practice Guidelines in the Veterans Health Administration
Hysong, S. J., Best, R. G., Pugh, J. A., & Moore, F. I. (2005) · Health Services Research · doi:10.1111/j.1475-6773.2005.00387.x · Cited by 62 · see citing works
Self-citationWahlström, R., Kounnavong, S., Sisounthone, B., Phanyanouvong, A., Southammavong, T., Eriksson, B., & Tomson, G. B. (2003) · Tropical Medicine & International Health · doi:10.1046/j.1365-3156.2003.01105.x · Cited by 61 · see citing works
Hysong, S. J., Best, R. G., & Pugh, J. A. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00610.x · Cited by 55 · see citing works
Self-citationMeterko, M., Young, G. J., White, B., et al. (2006) · Health Services Research · doi:10.1111/j.1475-6773.2006.00582.x · Cited by 49 · see citing works
Clustering estimates of effect magnitude from independent studies.
Hedges, L. V., & Olkin, I. (1983) · Psychological Bulletin · doi:10.1037/0033-2909.93.3.563 · Cited by 45 · see citing works
Robling, M. R., Houston, H. L. A., Kinnersley, P., Hourihan, M. D., Cohen, D. R., Hale, J., & Hood, K. (2002) · Clinical Radiology · doi:10.1053/crad.2001.0864 · Cited by 39 · see citing works
Knowledge of results and incorrect recall of plausible multiple-choice alternatives.
Karraker, R. J. (1967) · Journal of Educational Psychology · doi:10.1037/h0024113 · Cited by 17 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [16]Whitehead A Meta-analysis of Controlled Clinical Trials: Statistics in Practice Chichester, England John Wiley & Sons, Ltd. 2002
Cited by
Sorted by citations
Audit and feedback: effects on professional practice and healthcare outcomes
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., O'Brien, M. A., Johansen, M., Grimshaw, J., & Oxman, A. D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000259.pub3 · Cited by 5,485 · see citing works
Implementation strategies: recommendations for specifying and reporting
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-139 · Cited by 2,585 · see citing works
Ivers, N. M., Sales, A., Colquhoun, H., Michie, S., Foy, R., Francis, J. J., & Grimshaw, J. M. (2014) · Implementation Science · doi:10.1186/1748-5908-9-14 · Cited by 400 · see citing works
A systematic review of the use of theory in randomized controlled trials of audit and feedback
Colquhoun, H. L., Brehaut, J. C., Sales, A., Ivers, N., Grimshaw, J., Michie, S., Carroll, K., Chalifoux, M., & Eva, K. W. (2013) · Implementation Science · doi:10.1186/1748-5908-8-66 · Cited by 231 · see citing works
Gardner, B., Whittington, C., McAteer, J., Eccles, M. P., & Michie, S. (2010) · Social Science & Medicine · doi:10.1016/j.socscimed.2010.01.039 · Cited by 225 · see citing works
Refining a taxonomy for guideline implementation: results of an exercise in abstract classification
Mazza, D., Bairstow, P., Buchan, H., Chakraborty, S. P., Van Hecke, O., Grech, C., & Kunnamo, I. (2013) · Implementation Science · doi:10.1186/1748-5908-8-32 · Cited by 173 · see citing works
Building theories of knowledge translation interventions: Use the entire menu of constructs
Brehaut, J. C., & Eva, K. W. (2012) · Implementation Science · doi:10.1186/1748-5908-7-114 · Cited by 120 · see citing works
Implementing maternal death surveillance and response: a review of lessons from country case studies
Smith, H., Ameh, C., Roos, N., Mathai, M., & Broek, N. V. D. (2017) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-017-1405-6 · Cited by 107 · see citing works
Palinkas, L. A., Chou, C.-P., Spear, S. E., Mendon, S. J., Villamar, J., & Brown, C. H. (2020) · Implementation Science · doi:10.1186/s13012-020-01030-x · Cited by 70 · see citing works
Yoong, S. L., Nathan, N., Wolfenden, L., Wiggers, J., Reilly, K., Oldmeadow, C., Wyse, R., Sutherland, R., Delaney, T., Butler, P., et al. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0453-z · Cited by 52 · see citing works
- Authors
- Hysong, S. J.
- Affiliations
- United States Department of Veterans Affairs (US)
- Corresponding author
- Hysong, Sylvia J.
- ORCID
- Hysong, Sylvia J.
- Venue
- Medical Care
- Published
- 2009-03
- Volume
- 47
- Issue
- 3
- Pages
- 356-363
- ISSN
- 0025-7079 (ISSN-L)
- PMC
- PMC4170834
- OpenAlex
- W1964916989
- Cited by
- 306
- References
- 33
- Citations by year
- 2026: 7 · 2025: 15 · 2024: 17 · 2023: 12 · 2022: 19 · 2021: 22 · 2020: 17 · 2019: 13 · 2018: 18 · 2017: 23 · 2016: 24 · 2015: 25 · 2014: 23 · 2013: 23 · 2012: 21
- FWCI
- 29.05
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Medical Care (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- U.S. Department of Veterans Affairs (I01 HX003868)
- National Heart, Lung, and Blood Institute (R01HL079173)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Psychology › Applied Psychology › Behavioral Health and Interventions
- Keywords
- audit and feedback, feedback intervention theory, quality of care, feedback characteristics, written feedback, feedback frequency, goal setting, normative information, quality improvement intervention
- MeSH terms
- Feedback, Health Services Research, Humans, Medical Audit, Psychology, Industrial, Quality Assurance, Health Care, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
