Economic evaluations of audit and feedback interventions: a systematic review
Abstract
Background: The effectiveness of audit and feedback (A&F) interventions to improve compliance to healthcare guidelines is supported by randomised controlled trials (RCTs) and meta-analyses of RCTs. However, there is currently a knowledge gap on their cost-effectiveness. Objective: We aimed to assess whether A&F interventions targeting improvements in compliance to recommended care are economically favourable. Methods: We conducted a systematic review including experimental, observational and simulation-based economic evaluation studies of A&F interventions targeting healthcare providers. Comparators were a ‘do nothing’ strategy, or any other intervention not involving A&F or involving a subset of A&F intervention components. We searched MEDLINE, CINAHL, CENTRAL, Econlit, EMBASE, Health Technology Assessment Database, MEDLINE, NHS Economic Evaluation Database, ABI/INFORM, Web of Science, ProQuest and websites of healthcare quality associations to December 2021. Outcomes were incremental cost-effectiveness ratios, incremental cost-utility ratios, incremental net benefit and incremental cost-benefit ratios. Pairs of reviewers independently selected eligible studies and extracted relevant data. Reporting quality was evaluated using CHEERS (Consolidated Health Economic Evaluation Reporting Standards). Results were synthesised using permutation matrices for all studies and predefined subgroups. Results: Of 13 221 unique citations, 35 studies met our inclusion criteria. The A&F intervention was dominant (ie, at least as effective with lower cost) in 7 studies, potentially cost-effective in 26 and was dominated (ie, the same or less effectiveness and higher costs) in 2 studies. A&F interventions were more likely to be economically favourable in studies based on health outcomes rather than compliance to recommended practice, considering medical costs in addition to intervention costs, published since 2010, and with high reporting quality. Discussion: Results suggest that A&F interventions may have a high potential to be cost-effective. However, as is common in systematic reviews of economic evaluations, publication bias could have led to an overestimation of their economic value.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Much of the evidence for individual or groups of ERIC strategies remains narrowly focused, with prior systematic reviews and meta-analyses generally focusing on a specific strategy, subpopulation, disease, outcome, service setting, or geography.
“However, much of the evidence for individual or groups of ERIC strategies remains narrowly focused. Prior systematic reviews and meta-analyses have assessed strategy effectiveness, but have generally focused on a specific strategy, (e.g., Audit and Provide Feedback) [14–16], subpopulation, disease (e.g., individuals living with dementia) [16], outcome [15], service setting (e.g., primary care Clin”
From A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022 (2024) · cites this paper as [15]
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823 · see citing works
PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement
McGowan, J., Sampson, M., Salzwedel, D. M., Cogo, E., Foerster, V., & Lefebvre, C. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.021 · Cited by 5,700 · see citing works
Audit and feedback: effects on professional practice and healthcare outcomes
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., O'Brien, M. A., Johansen, M., Grimshaw, J., & Oxman, A. D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000259.pub3 · Cited by 5,485 · see citing works
Self-citationDe-duplication of database search results for systematic reviews in EndNote
Bramer, W. M., Giustini, D., de Jonge, G. B., Holland, L., & Bekhuis, T. (2016) · Journal of the Medical Library Association : JMLA · doi:10.3163/1536-5050.104.3.014 · Cited by 2,000 · see citing works
Evers, S., Goossens, M. E. J. B., de Vet, H. C. W., van Tulder, M. W., & Ament, A. J. H. A. (2005) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462305050324 · Cited by 860 · see citing works
Turner, L., Shamseer, L., Altman, D. G., Weeks, L., Peters, J., Kober, T., Dias, S., Schulz, K. F., Plint, A. C., & Moher, D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000030.pub2 · Cited by 740 · see citing works
Consolidated health economic evaluation reporting standards (CHEERS) statement
Husereau, D., Drummond, M., Petrou, S., Carswell, C., Moher, D., Greenberg, D., Augustovski, F., Briggs, A. H., Mauskopf, J., & Loder, E. (2013) · Value in Health · doi:10.1016/j.jval.2013.02.010 · Cited by 657 · see citing works
Assessing cost-effectiveness in healthcare: history of the $50,000 per QALY threshold
Grosse, S. D. (2008) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/14737167.8.2.165 · Cited by 637 · see citing works
Avery, A. J., Rodgers, S., Cantrill, J. A., Armstrong, S., Cresswell, K. M., Eden, M., Elliott, R. A., Howard, R., Kendrick, D., Morris, C. J., et al. (2012) · The Lancet · doi:10.1016/s0140-6736(11)61817-5 · Cited by 457 · see citing works
A web-based tool for adjusting costs to a specific target currency and price year
Shemilt, I., James, T., & Marcello, M. (2010) · Evidence & Policy · doi:10.1332/174426410x482999 · Cited by 411 · see citing works
Stetler, C., Legro, M., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., Hagedorn, H., Piñeros, S. L., & Wallace, C. (2006) · Implementation science : IS · doi:10.1186/1748-5908-1-23 · Cited by 392 · see citing works
Busse, R., Klazinga, N., Пантели, Д., et al. (2019) · Cited by 290 · see citing works
The quality of diagnostic accuracy studies since the STARD statement
Smidt, N., Rutjes, A. W., van der Windt, D., Ostelo, R., Bossuyt, P. M., Reitsma, J. B., Bouter, L. M., & de Vet, H. C. W. (2006) · Neurology · doi:10.1212/01.wnl.0000238386.41398.30 · Cited by 230 · see citing works
van Mastrigt, G. A. P. G., Hiligsmann, M., Arts, J. J. C., Broos, P. H., Kleijnen, J., Evers, S., & Majoie, M. (2016) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2016.1246960 · Cited by 218 · see citing works
Effectiveness of Decision Support for Families, Clinicians, or Both on HPV Vaccine Receipt
Fiks, A. G., Grundmeier, R. W., Mayne, S. L., Song, L., Feemster, K. A., Karavite, D., Hughes, C. C., Massey, J., Keren, R., Bell, L. M., et al. (2013) · PEDIATRICS · doi:10.1542/peds.2012-3122 · Cited by 215 · see citing works
Wijnen, B. F. M., van Mastrigt, G. A. P. G., Redekop, K., Majoie, M., de Kinderen, R. J. A., & Evers, S. (2016) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2016.1246961 · Cited by 214 · see citing works
Noritomi, D. T., Ranzani, O., Monteiro, M. B., Ferreira, E. M., Santos, S. R., Leibel, F., & Machado, F. R. (2013) · Intensive Care Medicine · doi:10.1007/s00134-013-3131-5 · Cited by 133 · see citing works
Mandrik, O. (., Severens, J. L. (., Bardach, A., Ghabri, S., Hamel, C., Mathes, T., Vale, L., Wisløff, T., & Goldhaber-Fiebert, J. D. (2021) · Value in Health · doi:10.1016/j.jval.2021.01.002 · Cited by 110 · see citing works
Quality of reporting of confounding remained suboptimal after the STROBE guideline
Pouwels, K. B., Widyakusuma, N. N., Groenwold, R. H. H., & Hak, E. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.08.009 · Cited by 103 · see citing works
Reporting and design elements of audit and feedback interventions: a secondary review
Colquhoun, H. L., Michie, S., Sales, A. E., Ivers, N. M., Grimshaw, J., Carroll, K. A., Chalifoux, M., Eva, K. W., & Brehaut, J. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-005004 · Cited by 98 · see citing works
Self-citationde la Perrelle, L., Radisic, G., Cations, M., Kaambwa, B., Barbery, G., & Laver, K. (2020) · BMC Health Services Research · doi:10.1186/s12913-020-4981-5 · Cited by 90 · see citing works
Summarising economic evaluations in systematic reviews: a new approach
Nixon, J., Khan, K. S., & Kleijnen, J. (2001) · BMJ · doi:10.1136/bmj.322.7302.1596 · Cited by 84 · see citing works
Dijkstra, R. F., Niessen, L. W., Braspenning, J. C. C., Adang, E., & Grol, R. T. P. M. (2006) · Diabetic Medicine · doi:10.1111/j.1464-5491.2005.01751.x · Cited by 81 · see citing works
Alali, A. S., Burton, K. R., Fowler, R., Naimark, D. M. J., Scales, D. C., Mainprize, T. G., & Nathens, A. B. (2015) · Value in Health · doi:10.1016/j.jval.2015.04.012 · Cited by 55 · see citing works
Huis, A. M. P., Hulscher, M., Adang, E., Grol, R. P. T. M., van Achterberg, T., & Schoonhoven, L. (2012) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2012.11.016 · Cited by 52 · see citing works
Wright, J., Bibby, J., Eastham, J., Harrison, S. A., McGeorge, M., Patterson, C., Price, N., Russell, D., Russell, I. T., Small, N., et al. (2007) · BMJ Quality & Safety · doi:10.1136/qshc.2006.019778 · Cited by 51 · see citing works
Barasa, E., Ayieko, P., Cleary, S., et al. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001238 · Cited by 51 · see citing works
Elliott, R. A., Putman, K. D., Franklin, M. K., Annemans, L., Verhaeghe, N., Eden, M., Hayre, J., Rodgers, S., Sheikh, A., & Avery, A. J. (2014) · PharmacoEconomics · doi:10.1007/s40273-014-0148-8 · Cited by 50 · see citing works
Naughton, C., Feely, J., & Bennett, K. (2009) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.01099.x · Cited by 49 · see citing works
Translating Pressure Ulcer Guidelines into Practice: It's Harder than It Sounds
Xakellis, G. C., Frantz, R. A., Lewis, A., et al. (2001) · Advances in Skin & Wound Care · doi:10.1097/00129334-200109000-00012 · Cited by 46 · see citing works
McRae, I., Butler, J. R., Sibthorpe, B. M., Ruscoe, W., Snow, J., Rubiano, D., & Gardner, K. (2008) · BMC Health Services Research · doi:10.1186/1472-6963-8-205 · Cited by 45 · see citing works
Morgan, T., Wu, J., Ovchinikova, L., Lindner, R. A., Blogg, S., & Moorin, R. E. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4773-y · Cited by 45 · see citing works
Fretheim, A., Aaserud, M., & Oxman, A. D. (2006) · PLoS Medicine · doi:10.1371/journal.pmed.0030216 · Cited by 43 · see citing works
Cost-effectiveness of integrated COPD care: the RECODE cluster randomised trial
Boland, M. R. S., Kruis, A. L., Tsiachristas, A., Assendelft, W. J., Gussekloo, J., Blom, C. M. G., Chavannes, N. H., & Mölken, M. R. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-007284 · Cited by 40 · see citing works
Implementing guidelines into clinical practice: what is the value?
Hoomans, T., Ament, A. J. H. A., Evers, S., & Severens, J. L. (2010) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2010.01557.x · Cited by 37 · see citing works
Davis, M. M., Nambiar, S., Mayorga, M. E., Sullivan, E., Hicklin, K., O’Leary, M. C., Dillon, K., Lich, K. H., Gu, Y., Lind, B. K., et al. (2019) · Preventive Medicine · doi:10.1016/j.ypmed.2019.105836 · Cited by 37 · see citing works
Wolf, M. S., Fitzner, K. A., Powell, E. F., McCaffrey, K. R., Pickard, A. S., McKoy, J. M., Lindenberg, J. H., Schumock, G. T., Carson, K. R., Ferreira, M. R., et al. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.02.6278 · Cited by 36 · see citing works
Claes, N., Moeremans, K., Buntinx, F., Jef, A., Jos, V., Herman, V. L., & Lieven, A. (2006) · Value in Health · doi:10.1111/j.1524-4733.2006.00129.x · Cited by 35 · see citing works
A cluster-randomized trial to reduce caesarean delivery rates in Quebec: cost-effectiveness analysis
Johri, M., Ng, E. S. W., Bermúdez‐Tamayo, C., Hoch, J. S., Ducruet, T., & Chaillet, N. (2017) · BMC Medicine · doi:10.1186/s12916-017-0859-8 · Cited by 34 · see citing works
Self-citationGuest, J. F., Keating, T., Gould, D. J., & Wigglesworth, N. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029971 · Cited by 33 · see citing works
Butler, A. E., Canamucio, A., Macpherson, D. S., Skoko, J. E., & True, G. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-013-2668-x · Cited by 28 · see citing works
Soleymani, F., Rashidian, A., Dinarvand, R., Kebriaeezade, A., Hosseini, M., & Abdollahi, M. R. (2012) · DARU Journal of Pharmaceutical Sciences · doi:10.1186/2008-2231-20-88 · Cited by 26 · see citing works
Model-Based Economic Evaluation of Treatments for Depression: A Systematic Literature Review
Kolovos, S., Bosmans, J. E., Riper, H. M., Chevreul, K., Coupé, V. M. H., & van Tulder, M. W. (2017) · PharmacoEconomics - Open · doi:10.1007/s41669-017-0014-7 · Cited by 26 · see citing works
Zhang, Z., Dawkins, B., Hicks, J. P., Walley, J. D., Hulme, C. T., Elsey, H., Deng, S., Lin, M., Zeng, J., & Wei, X. (2018) · Tropical Medicine & International Health · doi:10.1111/tmi.13132 · Cited by 24 · see citing works
Mewes, J. C., Steuten, L., IJsbrandy, C., IJzerman, M. J., & van Harten, W. H. (2017) · Value in Health · doi:10.1016/j.jval.2017.04.013 · Cited by 22 · see citing works
Robinson, M. B., Thompson, E. A., & Black, N. A. (1998) · BMJ Quality & Safety · doi:10.1136/qshc.7.1.19 · Cited by 20 · see citing works
Lara, C., Means, K. L., Morwood, K. D., Lighthall, W. R., Hoover, S., Tangka, F. K. L., French, C., Gayle, K. D., DeGroff, A. S., & Subramanian, S. (2018) · Cancer · doi:10.1002/cncr.31691 · Cited by 20 · see citing works
Strategies to reduce the use of low-value medical tests in primary care: a systematic review
Takada, T., Heus, P., van Doorn, S., Naaktgeboren, C. A., Weenink, J.-W., van Dulmen, S. A., & Hooft, L. (2020) · British Journal of General Practice · doi:10.3399/bjgp20x713693 · Cited by 20 · see citing works
López-Picazo, J. J., Ruiz, J. C., Sanchez, J. F., Ariza, Á., & Aguilera, B. (2011) · American Journal of Medical Quality · doi:10.1177/1062860610380898 · Cited by 17 · see citing works
When is it cost-effective to change the behaviour of health professionals ?
Mason, J. M., Freemantle, N., Nazareth, I., Eccles, M., Haines, A., & Drummond, M. (2001) · Durham Research Online (Durham University) · doi:10.1001/jama.286.23.2988 · Cited by 16 · see citing works
Callander, E. J., Creedy, D. K., Gamble, J. A., Fox, H., Toohill, J., Sneddon, A., & Ellwood, D. A. (2019) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12621 · Cited by 14 · see citing works
Lee, P., Zomer, E., & Liew, D. (2019) · Heart Lung and Circulation · doi:10.1016/j.hlc.2019.08.012 · Cited by 12 · see citing works
Ling, R., Giles, M., & Searles, A. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06871-w · Cited by 12 · see citing works
Navarro, H. J., Shakeshaft, A. P., Doran, C. M., & Petrie, D. (2012) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2012.01.007 · Cited by 11 · see citing works
Patel, B. N., Peiris, D., Patel, A. A., et al. (2020) · The Medical Journal of Australia · doi:10.5694/mja2.50667 · Cited by 11 · see citing works
Prospective ongoing prescribing error feedback to enhance safety: a randomised controlled trial
Gordon, M. A., & Jones, H. (2017) · Drugs & Therapy Perspectives · doi:10.1007/s40267-017-0412-z · Cited by 6 · see citing works
Yoo, M., Madaras‐Kelly, K., Nevers, M. R., Fleming-Dutra, K. E., Hersh, A. L., Ying, J., Haaland, B., Samore, M. H., & Nelson, R. E. (2021) · Infection Control and Hospital Epidemiology · doi:10.1017/ice.2021.393 · Cited by 6 · see citing works
Robinson, M. B., Thompson, E. A., & Black, N. A. (1998) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300010606 · Cited by 5 · see citing works
Wann lohnt es, eine medizinische Überversorgung abzubauen?
Gandjour, A., & Lauterbach, K. W. (2005) · Medizinische Klinik - Intensivmedizin und Notfallmedizin · doi:10.1007/s00063-005-1073-6 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Ashcraft, L. E., Goodrich, D. E., Hero, J., Phares, A., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Implementation Science · doi:10.1186/s13012-024-01369-5 · Cited by 89 · see citing works
- Authors
- Moore, L., Guertin, J. R., Tardif, P.-A., Ivers, N. M., Hoch, J., Conombo, B., Antony, J., Stelfox, H. T., Berthelot, S., Archambault, P., Turgeon, A., Gandhi, R., & Grimshaw, J.
- Affiliations
- CHU de Québec-Université Laval (CA)
- University of Toronto (CA)
- University of California, San Francisco (US)
- Women's College Hospital (CA)
- University of Calgary (CA)
- Children's Hospital of Eastern Ontario (CA)
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
- Corresponding author
- Moore, Lynne
- ORCID
- Venue
- BMJ Quality & Safety
- Publisher
- BMJ
- Published
- 2022-06-24
- Volume
- 31
- Issue
- 10
- Pages
- 754-767
- ISSN
- 1475-3898 (ISSN-L)
- 2044-5415
- 2044-5423
- OpenAlex
- W4283369394
- Cited by
- 33
- References
- 59
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 16 · 2023: 5
- FWCI
- 1.30
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- BMJ Quality & Safety (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Canadian Institutes of Health Research (168833, 353374)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- audit and feedback, cost-effectiveness, guideline adherence, healthcare workers, quality improvement intervention, economic evaluation, incremental cost-effectiveness ratio, publication bias, reporting quality
- MeSH terms
- Cost-Benefit Analysis, Delivery of Health Care, Feedback, Health Personnel, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
