Systematic review of economic evaluations and cost analyses of guideline implementation strategies
Open accessAbstract
To appraise the quality of economic studies undertaken as part of evaluations of guideline implementation strategies, we conducted a systematic review of such studies published between 1966 and 1998. Studies were assessed against BMJ economic evaluations guidelines for each stage (guideline development, implementation and treatment). Of 235 studies identified, 63 reported some information on cost. Only 3 studies provided evidence that their guideline was effective and efficient, 38 reported treatment costs only, 12 implementation and treatment costs, 11 implementation costs alone, and 2 guideline development, implementation and treatment costs. No study gave reasonably complete information on costs. Thus, very few satisfactory economic evaluations of guideline implementation strategies have been performed. Current evaluations have numerous methodological defects and rarely consider all relevant costs and benefits. Future evaluations should focus on evaluating the implementation of evidence-based guidelines.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Few studies include economic evaluations of implementation strategies.
“Few studies include economic evaluations of implementation strategies (87, 88).”
From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [87]
In a systematic review of 235 implementation studies, only 10% provided information about implementation costs.
“For example, in a systematic review of 235 implementation studies, only 10% provided information about implementation costs (87).”
From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [87]
Cites
Methods for the Economic Evaluation of Health Care Programmes
Drummond, M. E., Sculpher, M. J., Torrance, G. W., O’Brien, B. J., & Stoddart, G. L. (2005) · doi:10.1093/oso/9780198529446.001.0001 · Cited by 13,648 · see citing works
The National Institute for Clinical Excellence
(2001) · Nursing Management · doi:10.7748/nm.8.2.39.s15 · Cited by 3,868 · see citing works
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., Vale, L., Whitty, P., Eccles, M., Matowe, L., Shirran, L., et al. (2004) · Health Technology Assessment · doi:10.3310/hta8060 · Cited by 3,044 · see citing works
Self-citationEffect of clinical guidelines on medical practice: a systematic review of rigorous evaluations
Grimshaw, J., & Russell, I. T. (1993) · The Lancet · doi:10.1016/0140-6736(93)92244-n · Cited by 2,791 · see citing works
Self-citationOxman, A. D., Thomson, M., Davis, D. A., & Haynes, R. B. (1995) · PubMed Central · Cited by 1,847 · see citing works
Guidelines for authors and peer reviewers of economic submissions to the BMJ
Drummond, M. F., & Jefferson, T. O. (1996) · BMJ · doi:10.1136/bmj.313.7052.275 · Cited by 1,827 · see citing works
(1997) · Europe PMC (PubMed Central) · Cited by 1,226 · see citing works
Guidelines for Clinical Practice
Institute of Medicine (1992) · National Academies Press eBooks · doi:10.17226/1863 · Cited by 916 · see citing works
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J. M., Thomas, R. E., MacLennan, G., Fraser, C., Ramsay, C. R., Vale, L., Whitty, P., Eccles, M. P., Matowe, L., Shirran, L., et al. (2005) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462305290190 · Cited by 864 · see citing works
Self-citationImplementing guidelines and innovations in general practice: which interventions are effective?
Wensing, M., van der Weijden, T., & Grol, R. P. T. M. (1998) · Europe PMC (PubMed Central) · Cited by 417 · see citing works
Worrall, G. J., Chaulk, P. C., & Freake, D. (1997) · PubMed · Cited by 256 · see citing works
Feder, G. S., Griffiths, C., Highton, C., Eldridge, S. M., Spence, M., & Southgate, L. J. (1995) · BMJ · doi:10.1136/bmj.311.7018.1473 · Cited by 221 · see citing works
Reducing Vancomycin Use Utilizing a Computer Guideline: Results of a Randomized Controlled Trial
Shojania, K. G., Yokoe, D. S., Platt, R. G., Fiskio, J. M., Ma'Luf, N., & Bates, D. W. (1998) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1998.0050554 · Cited by 211 · see citing works
A Controlled Trial of Educational Outreach to Improve Blood Transfusion Practice
Soumerai, S. B. (1993) · JAMA · doi:10.1001/jama.1993.03510080065033 · Cited by 199 · see citing works
Barnett, G. O., Winickoff, R. N., Dorsey, J. L., Morgan, M. M., & Lurie, R. S. (1978) · Medical Care · doi:10.1097/00005650-197811000-00007 · Cited by 191 · see citing works
Guidelines in professions allied to medicine
Thomas, L. H., Cullum, N. A., McColl, E., Rousseau, N. S., Soutter, J. L., & Steen, N. (1999) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000349 · Cited by 183 · see citing works
Standardized clinical care pathways for major thoracic cases reduce hospital costs
Zehr, K. J., Dawson, P., Yang, S. C., et al. (1998) · The Annals of Thoracic Surgery · doi:10.1016/s0003-4975(98)00662-6 · Cited by 181 · see citing works
Weingarten, S. R., Riedinger, M. S., Conner, L., et al. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-120-4-199402150-00001 · Cited by 177 · see citing works
Truman, B. I., Smith-Akin, C. K., Hinman, A. R., Gebbie, K. M., Brownson, R. C., Novick, L. F., Lawrence, R. S., Pappaioanou, M., Fielding, J. E., Evans, C. A., et al. (2000) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(99)00124-5 · Cited by 177 · see citing works
Use of a Computer to Detect and Respond to Clinical Events: Its Effect on Clinician Behavior
McDonald, C. J. (1976) · Annals of Internal Medicine · doi:10.7326/0003-4819-84-2-162 · Cited by 169 · see citing works
Performance-based physician reimbursement and influenza immunization rates in the elderly
Kouides, R. W., Bennett, N. M., Lewis, B., Cappuccio, J. D., Barker, W. H., LaForce, F. M., & County, T. P.-C. P. O. M. (1998) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(97)00028-7 · Cited by 167 · see citing works
Improving Physician Performance Through Peer Comparison Feedback
Winickoff, R. N., Coltin, K. L., Morgan, M. M., Buxbaum, R. C., & Barnett, G. O. (1984) · Medical Care · doi:10.1097/00005650-198406000-00003 · Cited by 163 · see citing works
(1997) · PubMed · Cited by 147 · see citing works
Quality of Systematic Reviews of Economic Evaluations in Health Care
Jefferson, T. (2002) · JAMA · doi:10.1001/jama.287.21.2809 · Cited by 135 · see citing works
Effect of Population-Based Interventions on Laboratory Utilization
van Walraven, C., Goel, V., & Chan, B. (1998) · JAMA · doi:10.1001/jama.280.23.2028 · Cited by 129 · see citing works
Leviton, L. C., Goldenberg, R. L., Baker, C. S., Schwartz, R. M., Freda, M. C., Fish, L. J., Cliver, S. P., Rouse, D. J., Chazotte, C., Merkatz, I. R., et al. (1999) · JAMA · doi:10.1001/jama.281.1.46 · Cited by 128 · see citing works
Lee, T. H., Pearson, S. D., Johnson, P. A., et al. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-122-6-199503150-00006 · Cited by 126 · see citing works
Reduction of Incorrect Antibiotic Dosing Through a Structured Educational Order Form
Avorn, J. (1988) · Archives of Internal Medicine · doi:10.1001/archinte.1988.00380080024009 · Cited by 117 · see citing works
Changing professional practice: Theory and practice of clinical guidelines implementation
Thorsen, T., & Mākelä, M. (1999) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 115 · see citing works
Williams, A. (1974) · British Medical Bulletin · doi:10.1093/oxfordjournals.bmb.a071211 · Cited by 107 · see citing works
Computer-based vs manual health maintenance tracking. A controlled trial
Frame, P. (1994) · Archives of Family Medicine · doi:10.1001/archfami.3.7.581 · Cited by 104 · see citing works
Evaluation of a Pneumonia Practice Guideline in an Interventional Trial
Weingarten, S. R., Riedinger, M. S., Hobson, P., et al. (1996) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/ajrccm.153.3.8630553 · Cited by 100 · see citing works
Use of reminders for preventive procedures in family medicine.
Rosser, W. W., McDowell, I., & Newell, C. (1991) · PubMed · Cited by 97 · see citing works
Early Return to Work After Uncomplicated Myocardial Infarction
Dennis, C. A. (1988) · JAMA · doi:10.1001/jama.1988.03410020080032 · Cited by 96 · see citing works
Implementing preventive services. Success and failure in an outpatient trial
Belcher, D. W. (1990) · Archives of Internal Medicine · doi:10.1001/archinte.150.12.2533 · Cited by 95 · see citing works
Tools, teamwork, and tenacity: an office system for cancer prevention.
Carney, P. A., Dietrich, A. J., Keller, A., Landgraf, J. M., & Gt, O. (1992) · PubMed · Cited by 93 · see citing works
Inappropriate use of laboratory services: long term combined approach to modify request patterns.
Bareford, D., & Hayling, A. (1990) · BMJ · doi:10.1136/bmj.301.6764.1305 · Cited by 91 · see citing works
Marton, K. I. (1985) · Archives of Internal Medicine · doi:10.1001/archinte.1985.00360050060009 · Cited by 91 · see citing works
Integrated care for diabetes: clinical, psychosocial, and economic evaluation
for the Diabetes Integrated Care Evaluation Team (1994) · BMJ · doi:10.1136/bmj.308.6938.1208 · Cited by 85 · see citing works
Sculpher, M. J. (2000) · Family Practice · doi:10.1093/fampra/17.suppl_1.s26 · Cited by 83 · see citing works
Keyserling, T. C. (1997) · Archives of Family Medicine · doi:10.1001/archfami.6.2.135 · Cited by 75 · see citing works
Changing antibiotic prescribing by educational marketing
Landgren, F. T., Harvey, K. J., Moulds, R. F. W., et al. (1988) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.1988.tb120797.x · Cited by 74 · see citing works
Bryce, F. P., Neville, R. G., Crombie, I. K., Clark, R. A., & McKenzie, P. (1995) · BMJ · doi:10.1136/bmj.310.6983.838 · Cited by 73 · see citing works
Improving the Utilization of Screening Mammography in Primary Care Practices
Nattinger, A. B. (1989) · Archives of Internal Medicine · doi:10.1001/archinte.1989.00390090121024 · Cited by 68 · see citing works
Everitt, D. E., Soumerai, S. B., Avorn, J., et al. (1990) · Infection Control and Hospital Epidemiology · doi:10.1086/646098 · Cited by 64 · see citing works
The effect of provider education on blood utilization practices
Morrison, J. C., Sumrall, D. D., Chevalier, S. P., et al. (1993) · American Journal of Obstetrics and Gynecology · doi:10.1016/0002-9378(93)90289-u · Cited by 60 · see citing works
Guideline-based Consultation To Prevent Anticoagulant-related Bleeding
Landefeld, C. S., & Anderson, P. A. (1992) · Annals of Internal Medicine · doi:10.7326/0003-4819-116-10-829 · Cited by 57 · see citing works
Lobach, D. F. (1996) · Europe PMC (PubMed Central) · Cited by 56 · see citing works
Aid to diagnosis of melanoma in primary medical care
Mar, C. D., & Green, A. C. (1995) · BMJ · doi:10.1136/bmj.310.6978.492 · Cited by 55 · see citing works
Early "Step-Down" Transfer of Low-Risk Patients with Chest Pain
Weingarten, S. R., Ermann, B., Bolus, R., et al. (1990) · Annals of Internal Medicine · doi:10.7326/0003-4819-113-4-283 · Cited by 51 · see citing works
Bloor, K. E., Freemantle, N., Khadjesari, Z., et al. (2003) · BMJ · doi:10.1136/bmj.326.7389.578 · Cited by 48 · see citing works
Fergusson, D. A., vanWalraven, C., Coyle, D., Laupacis, A., & THEINTERNATIONALSTUDYOFPERIO (1999) · Transfusion Medicine Reviews · doi:10.1016/s0887-7963(99)80005-4 · Cited by 46 · see citing works
Medicare Reimbursement for Preventive Care
MORRISSEY, J. P., Harris, R., Norburn, J. E. K., McLaughlin, C. P., Garrett, J. M., Jackman, A., Stein, J. S., Lannon, C., Schwartz, R. J., Patrick, D. L., et al. (1995) · Medical Care · doi:10.1097/00005650-199504000-00001 · Cited by 44 · see citing works
Effect of Departmental Policies on Cesarean Delivery Rates: A Community Hospital Experience
Poma, P. A. (1998) · Obstetrics and Gynecology · doi:10.1016/s0029-7844(98)00077-5 · Cited by 44 · see citing works
A Successful Experiment to Reduce Unnecessary Laboratory Use in a Community Hospital
Gortmaker, S. L., BICKFORD, A. F., Mathewson, H. O., DUMBAUGH, K., & TIRRELL, P. C. (1988) · Medical Care · doi:10.1097/00005650-198806000-00011 · Cited by 42 · see citing works
Computerized concurrent review of prenatal care.
Morgan, M., Studney, D. R., Go, B., & Winickoff, R. N. (1978) · PubMed · Cited by 41 · see citing works
Improving antiulcer agent prescribing in a health maintenance organization
Raisch, D. W., Bootman, J. L., Larson, L. N., et al. (1990) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/47.8.1766 · Cited by 40 · see citing works
Effect of Clinical Pharmacy Services on Prescribing on an Orthopedic Unit
Herfindal, E. T., Bernstein, L. R., & Kishi, D. T. (1983) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/40.11.1945 · Cited by 39 · see citing works
The effect on cost of medical care for patients treated with an automated clinical audit system
Thomas, J. C., Moore, A. C., & Qualls, P. E. (1983) · Journal of Medical Systems · doi:10.1007/bf00993294 · Cited by 39 · see citing works
A prospective controlled trial of computerized decision support for lipid management in primary care
Hobbs, R. F., Delaney, B. C., Carson, A., & Kenkre, J. E. (1996) · Family Practice · doi:10.1093/fampra/13.2.133 · Cited by 39 · see citing works
Limitations of provider interventions in hypertension quality assurance.
Winickoff, R. N., Wilner, S., Neisuler, R., & Barnett, G. O. (1985) · American Journal of Public Health · doi:10.2105/ajph.75.1.43 · Cited by 35 · see citing works
Designing Evaluations of Interventions to Change Professional Practice
Mason, J. M., Wood, J., & Freemantle, N. (1999) · Journal of Health Services Research & Policy · doi:10.1177/135581969900400209 · Cited by 35 · see citing works
Jones, R. H., LYDEARD, S., & Dunleavey, J. (1993) · BMJ Quality & Safety · doi:10.1136/qshc.2.4.217 · Cited by 33 · see citing works
Nursing Home‐Acquired Pneumonia: Outcomes from a Clinical Process Improvement Program
Dempsey, C. L. (1995) · Pharmacotherapy The Journal of Human Pharmacology and Drug Therapy · doi:10.1002/j.1875-9114.1995.tb04345.x · Cited by 32 · see citing works
Clarke, J. A., & Adams, J. E. (1990) · Clinical Radiology · doi:10.1016/s0009-9260(05)80957-2 · Cited by 32 · see citing works
Dranitsaris, G., Warr, D. G., & Puodziunas, A. (1995) · Supportive Care in Cancer · doi:10.1007/bf00368888 · Cited by 31 · see citing works
Shifting Physician Prescribing to a Preferred Histamine-2-Receptor Antagonist
Brufsky, J. W., Ross‐Degnan, D., Calabrese, D., Gao, X., & Soumerai, S. B. (1998) · Medical Care · doi:10.1097/00005650-199803000-00009 · Cited by 31 · see citing works
Reducing the Use of H 2 ‐Receptor Antagonists in the Long‐Term‐Care Setting
Gurwitz, J. H., Noonan, J. P., & Soumerai, S. B. (1992) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1992.tb02135.x · Cited by 29 · see citing works
Cost Savings Using a Stepped-Care Prescribing Protocol for Nonsteroidal Anti-inflammatory Drugs
Jones, D. L. (1996) · JAMA · doi:10.1001/jama.1996.03530360036035 · Cited by 27 · see citing works
Should physicians perform their own quality assurance audits?
Brady, W. J., Hissa, D. C., McConnell, M., & Wones, R. G. (1988) · Journal of General Internal Medicine · doi:10.1007/bf02596100 · Cited by 25 · see citing works
Stuart, M. E., Macuiba, J., Heidrich, F., et al. (1997) · PubMed · Cited by 20 · see citing works
When is it cost-effective to change the behaviour of health professionals ?
Mason, J. M., Freemantle, N., Nazareth, I., Eccles, M., Haines, A., & Drummond, M. (2001) · Durham Research Online (Durham University) · doi:10.1001/jama.286.23.2988 · Cited by 16 · see citing works
Palmer, R. H., & Hargraves, J. L. (1996) · Medical Care · doi:10.1097/00005650-199609002-00010 · Cited by 16 · see citing works
Effect of a selective restriction policy on antibiotic expenditure and use: an institutional model.
Suwangool, P., Moola-Or, P., Waiwatana, A., et al. (1991) · PubMed · Cited by 15 · see citing works
Changing Physician Behavior in Ordering Digoxin Assays
Fraser, G. L., Wennberg, D. E., Dickens, J. D., et al. (1996) · Annals of Pharmacotherapy · doi:10.1177/106002809603000502 · Cited by 13 · see citing works
Pe, M., & Gartenberg, G. (1985) · PubMed · Cited by 12 · see citing works
Effect of a pharmacy-based cost-containment program on cimetidine costs and use
Kong, K. M., Johnson, W. F., Anderson, P. O., et al. (1987) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/44.9.2068 · Cited by 8 · see citing works
Implementation and Evaluation of a Computerized Reminder System in Ambulatory Care
Banks, N. J., Palmer, R. H., Kane, N., et al. (1988) · PubMed Central · Cited by 7 · see citing works
Pearce, M. J., & Ej, B. (1997) · PubMed · Cited by 7 · see citing works
An evaluation of a guideline based open access urological investigation service.
Thomas, R. E. (2002) · Europe PMC (PubMed Central) · Cited by 4 · see citing works
Self-citationThe development, implementation and evaluation of prescribing guidelines in general practice.
Mc, W. (1998) · Europe PMC (PubMed Central) · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda
Powell, B. J., Fernandez, M. E., Williams, N. J., Aarons, G. A., Beidas, R. S., Lewis, C. C., McHugh, S. M., & Weiner, B. J. (2019) · Frontiers in Public Health · doi:10.3389/fpubh.2019.00003 · Cited by 775 · see citing works
A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions
Powell, B. J., Proctor, E. K., & Glass, J. E. (2014) · Research on Social Work Practice · doi:10.1177/1049731513505778 · Cited by 268 · see citing works
Eisman, A. B., Kilbourne, A. M., Dopp, A. R., Saldana, L., & Eisenberg, D. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2019.06.008 · Cited by 210 · see citing works
Expert recommendations for implementing change (ERIC): protocol for a mixed methods study
Waltz, T. J., Powell, B. J., Chinman, M. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., Damschroder, L. J., & Kirchner, J. E. (2014) · Implementation Science · doi:10.1186/1748-5908-9-39 · Cited by 202 · see citing works
Economic evaluation of implementation strategies in health care
Hoomans, T., & Severens, J. L. (2014) · Implementation Science · doi:10.1186/s13012-014-0168-y · Cited by 181 · see citing works
Mortimer, D., French, S. D., McKenzie, J. E., O′Connor, D. A., & Green, S. E. (2013) · PLoS ONE · doi:10.1371/journal.pone.0075647 · Cited by 30 · see citing works
Rycroft-Malone, J., Gradinger, F., Griffiths, H. O., Crane, R., Gibson, A., Mercer, S., Anderson, R., & Kuyken, W. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05140 · Cited by 24 · see citing works
Moretto, N., Comans, T. A., Chang, A. T., O’Leary, S. P., Osborne, S., Carter, H. E., Smith, D., Cavanagh, T., Blond, D., & Raymer, M. (2019) · Implementation Science · doi:10.1186/s13012-019-0923-1 · Cited by 25 · see citing works
- Authors
- Vale, L., Thomas, R., MacLennan, G., & Grimshaw, J.
- Affiliations
- University of Aberdeen (GB)
- Corresponding author
- Vale, Luke
- Venue
- The European Journal of Health Economics
- Published
- 2007-03-09
- Volume
- 8
- Issue
- 2
- Pages
- 111-121
- ISSN
- 1439-3972 (ISSN-L)
- 1618-7598
- 1618-7601
- OpenAlex
- W2093313132
- Cited by
- 107
- References
- 82
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 10 · 2023: 6 · 2022: 6 · 2021: 5 · 2020: 5 · 2019: 7 · 2018: 2 · 2017: 4 · 2016: 3 · 2015: 12 · 2014: 4 · 2013: 10 · 2012: 4
- FWCI
- 2.44
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- The European Journal of Health Economics (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Aberdeen University Research Archive (Aberdeen University) (submittedVersion)
- Newcastle University ePrints (Newcastle Univesity)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Health Sciences › Health Professions › General Health Professions › Healthcare cost, quality, practices
- Keywords
- economic evaluation, cost analysis, guideline implementation, evidence-based recommendations, implementation costs, treatment costs
- MeSH terms
- Costs and Cost Analysis, Practice Guidelines as Topic, Guideline Adherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
