Use of health economic evaluation in the implementation and improvement science fields—a systematic literature review
Open accessAbstract
BACKGROUND: Economic evaluation can inform whether strategies designed to improve the quality of health care delivery and the uptake of evidence-based practices represent a cost-effective use of limited resources. We report a systematic review and critical appraisal of the application of health economic methods in improvement/implementation research. METHOD: A systematic literature search identified 1668 papers across the Agris, Embase, Global Health, HMIC, PsycINFO, Social Policy and Practice, MEDLINE and EconLit databases between 2004 and 2016. Abstracts were screened in Rayyan database, and key data extracted into Microsoft Excel. Evidence was critically appraised using the Quality of Health Economic Studies (QHES) framework. RESULTS: Thirty studies were included-all health economic studies that included implementation or improvement as a part of the evaluation. Studies were conducted mostly in Europe (62%) or North America (23%) and were largely hospital-based (70%). The field was split between improvement (N = 16) and implementation (N = 14) studies. The most common intervention evaluated (43%) was staffing reconfiguration, specifically changing from physician-led to nurse-led care delivery. Most studies (N = 19) were ex-post economic evaluations carried out empirically-of those, 17 were cost effectiveness analyses. We found four cost utility analyses that used economic modelling rather than empirical methods. Two cost-consequence analyses were also found. Specific implementation costs considered included costs associated with staff training in new care delivery pathways, the impacts of new processes on patient and carer costs and the costs of developing new care processes/pathways. Over half (55%) of the included studies were rated 'good' on QHES. Study quality was boosted through inclusion of appropriate comparators and reporting of incremental analysis (where relevant); and diminished through use of post-hoc subgroup analysis, limited reporting of the handling of uncertainty and justification for choice of discount rates. CONCLUSIONS: The quantity of published economic evaluations applied to the field of improvement and implementation research remains modest; however, quality is overall good. Implementation and improvement scientists should work closely with health economists to consider costs associated with improvement interventions and their associated implementation strategies. We offer a set of concrete recommendations to facilitate this endeavour.
AI snapshot
Written from the paper’s full text.
TL;DR
This systematic review identified 30 studies (2004–2016) applying health economic evaluation to implementation and improvement science, finding modest quantity but overall good quality. It recommends closer collaboration with health economists and better capture of implementation/hidden costs, plus greater use of ex-ante and hybrid evaluative designs.
Key findings
- From 1668 records, 30 studies were included: 16 improvement and 14 implementation studies. Most were European (62%) or North American (23%) and hospital-based (70%).
- The most common intervention was staffing reconfiguration in 13 studies (43%), specifically changing from physician-led to nurse-led care. Nineteen studies were ex-post empirical economic evaluations, including 17 cost-effectiveness analyses; four cost-utility analyses used economic modelling, and two were cost-consequence analyses.
- QHES appraisal covered 22 papers: average score was 76/100, 13 studies (62%) scored >75, and one scored 100. The abstract reports over half (55%) were rated “good.” Improvement studies performed better than implementation studies.
- Only six of 30 studies explicitly assessed implementation-related “hidden” costs such as training, development, and monitoring; three included rollout/scaling scenarios, and one considered funding sources.
Methods
- Systematic literature search across Agris, Embase, Global Health, HMIC, PsycINFO, Social Policy and Practice, MEDLINE, and EconLit, covering publications after 2003 through 16 March 2016; 1668 papers were screened in Rayyan and extracted into Microsoft Excel.
- Inclusion required English-language completed studies in public health/primary/secondary/tertiary care with implementation or improvement aspects, quantifiable patient/service outcomes, and specified economic methods; cost-only studies, protocols, methodological papers, and conference abstracts were excluded.
- Methodological quality was assessed with the Quality of Health Economic Studies (QHES) framework, scoring 16 criteria out of 100, with ≥75 considered “good.”
Limitations
- The authors state that strict inclusion criteria may have limited the number of studies identified and synthesised; they excluded cost-only studies and non-English papers, making criteria stricter than prior reviews.
- The authors state they cannot rule out whether early-phase economic analyses are not being done or are simply not reported in academic publications.
- Evident from the text: QHES appraisal excluded literature reviews, meta-analyses, and commentaries, so quality conclusions apply to 22 of 30 included papers; comparison with Hoomans et al. is limited because the two reviews used different quality appraisal frameworks.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 34 cited sources checked · 0 supported · 0 flagged
- Findingown data
Thirty studies were included in the final evidence review and synthesis.
“Thirty studies were included in the final evidence review and synthesis.”
- Also stated in: § Results · plain
“Thirty studies were included—all health economic studies that included implementation or improvement as a part of the evaluation.”
Quote in the paperSubstantial claim - Also stated in: § Results · plain
- Findingown data
The most common intervention type, evaluated in 13 (43%) of the included studies, was staffing reconfiguration for service quality improvement, specifically changing from physician-led to nurse-led delivery of interventions to patients.
“The most common intervention type, evaluated in 13 (43%) of the included studies, was staffing reconfiguration for service quality improvement, specifically changing from physician-led to nurse-led delivery of interventions to patients.”
- Also stated in: § Results · plain
“The most common intervention evaluated (43%) was staffing reconfiguration, specifically changing from physician-led to nurse-led care delivery.”
Quote in the paperNot a substantial claim - Also stated in: § Results · plain
- Findingown data
Interventions involving general service reorganisation or changes to existing systems of care were the primary focus in ten (33%) of the reviewed studies.
“More broadly, interventions involving general service reorganisation or changes to existing systems of care were the primary focus in ten (33%) of studies reviewed.”
Quote in the paperNot a substantial claim - Findingown data
Of the 30 included studies, 6 (23%) were based in either the USA or Canada.
“six studies (23%) were based in either the USA or Canada”
- Also stated in: § Results · plain
“or North America (23%)”
Quote in the paperNot a substantial claim - Also stated in: § Results · plain
- Findingown data
21 of the 30 included studies were hospital-based.
“In terms of health care settings, 21 studies were hospital-based”
- Also stated in: § Results · plain
“and were largely hospital-based (70%)”
Quote in the paperNot a substantial claim - Also stated in: § Results · plain
- Findingnone given
All these ex-post evaluations compared a new intervention against current practice.
“All these evaluations compared a new intervention against current practice.”
Quote in the paperNot a substantial claim - Findingown data
The most common focus of the reviewed improvement studies was staff reconfigurations within a clinical area from medical to nursing staff.
“The most common focus of the reviewed improvement studies was staff reconfigurations within a clinical area from medical to nursing staff”
Quote in the paperNot a substantial claim - Findingown data
The most common focus of the reviewed implementation studies was implementation strategies of new care pathways or novel services.
“for implementation studies, the most common focus was on implementation strategies of new care pathways or novel services.”
Quote in the paperNot a substantial claim - Findingown data
The best performing QHES dimensions were the methodological dimensions.
“The best performing QHES dimensions were the methodological dimensions.”
Quote in the paperNot a substantial claim - Findingown data
Comparison between economic studies identified in a previous review by Hoomans et al. (1998-2004) and those in this review (2004-2016) shows evidence of a general improvement in quality over the past two decades.
“Comparison between economic studies identified in a previous review carried out by Hoomans et al. (covering the immediately preceding period 1998 to 2004) with those identified in this review (2004 to 2016) shows evidence of a general improvement in quality over the past two decades”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Hoomans et al.
Not checked in the free run.
- Findingown data
Incremental analysis with a relevant comparator (QHES dimension 6) was used in all but one study.
“Incremental analysis with a relevant comparator (dimension 6) was used in all but one study”
Quote in the paperNot a substantial claim - Findingown data
Justification for chosen discount rates was not provided in around half the studies that used them.
“Justification for chosen discount rates was not provided in around half the studies that used them.”
Quote in the paperNot a substantial claim - Findingown data
Many of the economic studies picked up in the review were linked to wider studies built around more traditional evaluative research designs, specifically randomised controlled trials.
“many of the economic studies picked up in our review were linked to wider studies built around more traditional evaluative research designs, specifically randomised controlled trials.”
Quote in the paperSubstantial claim - Findingnone given
Discount rates were correctly applied and stated when adjusting for timing of costs and benefits in all cases where measured costs and outcomes extended beyond 1 year.
“Discount rates were correctly applied and stated when adjusting for timing of costs and benefits in all cases where measured costs and outcomes extended beyond 1 year.”
Quote in the paperNot a substantial claim - Findingown data
There were aspects of improvement and implementation that were not adequately covered in many studies.
“we found that there were aspects of improvement and implementation that were not adequately covered in many studies.”
Quote in the paperSubstantial claim - Findingown data
Thirteen of the studies (62%) attained a 'good' score of over 75 on QHES.
“Thirteen of the studies (62%) attained a ‘good’ score of over 75.”
Quote in the paperNot a substantial claim - Findingown data
The studies evaluated costs and outcomes retrospectively using data over a period of time following the introduction of a specific improvement or implementation initiative.
“The studies evaluated costs and outcomes retrospectively using data over a period of time following the introduction of a specific improvement or implementation initiative.”
Quote in the paperSubstantial claim - Findingcitation
Costing methodology was only deemed adequate in 11% of cases included in the Hoomans et al. review, compared to 76% of the studies in this review.
“costing methodology was only deemed adequate in 11% of cases included in the Hoomans et al. review, compared to 76% of the studies in this review.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Hoomans et al.
Not checked in the free run.
- Findingown data
Justification for the use of effectiveness measures and study outcomes was given in two-thirds of studies.
“there was justification for the use of effectiveness measures and study outcomes given in two-thirds of studies.”
Quote in the paperNot a substantial claim - Findingown data
Three quarters of studies gave details of what methodology was used to quantify service inputs and the sources and methods used for estimating unit costs.
“While three quarters of studies gave details of what methodology was used to quantify service inputs (such as use of self-report service use schedules) and the sources and methods used for estimating unit costs”
Quote in the paperNot a substantial claim - Findingcitation
There was no evidence that economic methods have as yet been integrated into more advanced evaluative designs within the fields of improvement and implementation design, particularly 'hybrid' designs.
“There was no evidence that economic methods have as yet been integrated into more advanced evaluative designs within the fields of improvement and implementation design, particularly ‘hybrid’ designs”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [38] Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217. source
Not checked in the free run.
- cites [39] Brown CH, Curran G, Palinkas LA, Aarons GA, Wells KB, Jones L, Collins LM, Duan N, Mittman BS, Wallace A, Tabak RG. An overview of research and evaluation designs for dissemination and implementation. Annu Rev Public Health. 2017;38:1–22. source
Not checked in the free run.
- Findingown data
Specific implementation costs (training staff in new care delivery pathways, impacts of new processes on patient and carer costs, and costs of developing new processes) were considered by six of the reviewed studies.
“Specific implementation costs, such as those associated with training staff in new care delivery pathways, the impacts of new processes on patient and carer costs and the costs of developing the new processes were considered by six of the reviewed studies.”
- Also stated in: § Results · plain
“Specific implementation costs considered included costs associated with staff training in new care delivery pathways, the impacts of new processes on patient and carer costs and the costs of developing new care processes/pathways.”
Quote in the paperNot a substantial claim - Also stated in: § Results · plain
- Findingown data
Study quality was boosted through inclusion of appropriate comparators and reporting of incremental analysis (where relevant).
“Study quality was boosted through inclusion of appropriate comparators and reporting of incremental analysis (where relevant)”
Quote in the paperNot a substantial claim - Findingown data
The costing element, covered by QHES dimension 9, performed poorly overall.
“The costing element, covered by dimension 9, performed poorly overall.”
Quote in the paperNot a substantial claim - Findingnone given
Improvement studies performed overall better than implementation studies on the QHES.
“Improvement studies performed overall better than implementation studies on the QHES.”
Quote in the paperNot a substantial claim - Findingnone given
Where subgroup analysis was carried out, this was done post-hoc rather than being pre-planned with a clear a priori justification for the use of the chosen subgroups.
“Where subgroup analysis was carried out, this was done post-hoc rather than being pre-planned with a clear a priori justification for the use of the chosen subgroups.”
Quote in the paperNot a substantial claim - Findingown data
Only six out of 30 studies included an explicit assessment of the 'hidden' costs of improvement and implementation strategies.
“Only six out of 30 studies included an explicit assessment of these type of ‘hidden’ costs of improvement and implementation strategies.”
Quote in the paperSubstantial claim - Findingown data
A little over a quarter of the included studies declared the perspective of their analysis and gave a justification for the perspective used.
“A little over a quarter of the included studies declared the perspective of their analysis and gave a justification for the perspective used.”
Quote in the paperNot a substantial claim - Findingown data
The review demonstrates an increasing number of health economic evaluations nested within implementation and improvement research studies.
“Our review demonstrates an increasing number of health economic evaluations nested within implementation and improvement research studies”
Quote in the paperSubstantial claim - Findingown data
Only one study obtained a 'perfect' QHES score of 100 points.
“Only one study [33] obtained a ‘perfect’ score of 100 points.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [33] Afzali HH, Gray J, Beilby J, Holton C, Karnon J. A model-based economic evaluation of improved primary care management of patients with type 2 diabetes in Australia. Appl Health Econ Health Policy. 2013;11(6):661–70. source
Not checked in the free run.
- Findingown data
Only a third of studies gave details of how parameter uncertainty was addressed in relation to the study conclusions.
“Only a third gave details of how parameter uncertainty was addressed in relation to the study conclusions.”
Quote in the paperNot a substantial claim - Findingown data
The quality of economic evidence in the field of improvement and implementation science has improved over time.
“Our systematic review of the recent literature has shown that the quality of economic evidence in the field of improvement and implementation science has improved over time”
Quote in the paperSubstantial claim - Findingown data
In 81% of studies, the data sources for the analysis were from randomised controlled trials.
“in 81% of studies the data sources for the analysis were from randomised controlled trials”
Quote in the paperNot a substantial claim - Findingown data
The health economic evaluations nested within implementation and improvement research studies appear to be improving in methodological quality in recent years.
“which further appear to be improving in methodological quality in recent years.”
Quote in the paperNot a substantial claim - Findingown data
Twenty-two of the papers were included in the QHES economic quality appraisal.
“Twenty-two of the papers were included in the QHES economic quality appraisal”
Quote in the paperNot a substantial claim - Findingown data
The average QHES quality score was 76 out of a possible 100.
“The average quality score was 76 out of a possible 100 (Fig. 2).”
Quote in the paperNot a substantial claim - Findingown data
Over half (55%) of the included studies were rated 'good' on QHES.
“Over half (55%) of the included studies were rated ‘good’ on QHES.”
Quote in the paperNot a substantial claim - Findingown data
Such earlier phase economic analyses were simply not found in the review.
“Such earlier phase economic analyses were simply not found in our review.”
Quote in the paperSubstantial claim - Findingown data
Only two studies gave justification for why they chose the method used to quantify service inputs and estimate unit costs.
“only two gave justification for why they chose that method.”
Quote in the paperNot a substantial claim - Premisecitation
Two earlier reviews (Hoomans et al. 2007; Grimshaw et al. 2004) found evidence of limited quality and scope for understanding the cost-effectiveness of implementation strategies.
“In two earlier reviews (Hoomans et al. in 2007 [3] and earlier Grimshaw et al. in 2004 [4]), the use of economic methods in evaluating the implementation of evidence-based guidelines was examined, and the authors found evidence of limited quality and scope for understanding the cost-effectiveness of implementation strategies.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [3] Hoomans T, Evers SMAA, Ament AJHA, Hübben MWA, Van Der Weijden T, Grimshaw JM, et al. The methodological quality of economic evaluations of guideline implementation into clinical practice: a systematic review of empiric studies. Value Health. 2007;10(4):305–16. source
Not checked in the free run.
- cites [4] Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Int J Technol Assess Health Care. 2005;21(1):149. source
Not checked in the free run.
- Premisecitation
The application of economic evaluation within improvement and implementation evaluative research remains relatively limited.
“its application within improvement and implementation evaluative research remains relatively limited [1].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [1] Hoomans T, Severens JL. Economic evaluation of implementation strategies in health care. Implement Sci. 2014;9(1):168. source
Not checked in the free run.
- Interpretationown data
The quantity of published economic evaluations applied to the field of improvement and implementation research remains modest.
“The quantity of published economic evaluations applied to the field of improvement and implementation research remains modest”
Quote in the paperSubstantial claim - Study design and conductreasoning
The inclusion criteria were strict, which may have limited the number of studies that were identified and synthesised.
“our inclusion criteria were strict, which may have limited the number of studies that we identified and synthesised.”
Quote in the paperNot a substantial claim - Interpretationown data
The reviewed studies were generally of good quality.
“The reviewed studies were generally of good quality.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Economic analysis, and particularly economic modelling, currently at least appears to have a less important role in informing decisions over which options to pursue at earlier stages of implementing change, and in the appraisal of spread and scale up within wider populations.
“it also suggests that economic analysis, and particularly economic modelling, currently at least appears to have a less important role in informing decisions over which options to pursue at earlier stages of implementing change, and in the appraisal of spread and scale up within wider populations.”
Quote in the paperSubstantial claim - Interpretationreasoning
The improvement in quality is attributed at least partly to improvements in reporting economic analyses over time.
“We attribute it at least partly to improvements in reporting economic analyses over time, which would appear to have made an impact on the studies we captured.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 19
- May combine two claims. 17
- The quote only partly states this claim. 1
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataFour included studies were from Australia, and one each from Ethiopia, a subset of African countries (Uganda, Kenya and South Africa) and Malaysia.
“four from Australia and one each from Ethiopia, a subset of African countries (Uganda, Kenya and South Africa) and Malaysia.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOf the 30 included studies, 17 (62%) were European-based, mostly from the UK (12 studies).
“Seventeen of the studies (62%) were European-based (mostly from the UK—12 studies)”
- Also stated in: § Results · plain
“Studies were conducted mostly in Europe (62%)”
Quote in the paperNot a substantial claim- Also stated in: § Results · plain
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe initial search strategy identified 1668 articles, of which 1566 were excluded (1525 during initial screen and 41 following full text screening).
“In total, the initial search strategy identified 1668 articles, of which 1566 were excluded, 1525 during the initial screen and 41 following full text screening.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataSixteen of the included studies were identified as improvement studies and 14 as implementation studies.
“Sixteen of the included studies were identified as ‘improvement’ studies (see Table 3, panel 1a) and 14 were identified as ‘implementation’ studies (see Table 3, panel 1b).”
- Also stated in: § Results · plain
“The field was split between improvement (N = 16) and implementation (N = 14) studies.”
Quote in the paperNot a substantial claim- Also stated in: § Results · plain
May combine two claims.
Findingown dataThere were four further cost-utility analyses that used economic modelling rather than empirical methods, and two cost-consequence analyses.
“There were also four further CUAs that used economic modelling rather than empirical methods [8–10, 34], and two cost-consequence analyses [16, 35].”
- Also stated in: § Results · plain
“We found four cost utility analyses that used economic modelling rather than empirical methods. Two cost-consequence analyses were also found.”
Quote in the paperNot a substantial claim0 of 6 sources checked- cites [8] Burr JM, Mowatt G, Hernández R, Siddiqui MA, Cook J, Lourenco T, Ramsay C, Vale L, Fraser C, Azuara-Blanco A, Deeks J. The clinical effectiveness and cost-effectiveness of screening for open angle glaucoma: a systematic review and economic evaluation. Health Technol Assess (Winch. Eng.). 2007;11(41):1–90.
Not checked in the free run.
- cites [9] Burr JM, Botello-Pinzon P, Takwoingi Y, Hernández R, Vazquez-Montes M, Elders A, Asaoka R, Banister K, van der Schoot J, Fraser C, King A, Lemij H, Sanders R, Vernon S, Tuulonen A, Kotecha A, Glasziou P, Garway-Heath D, Crabb D, Vale L, Azuara-Blanco A, Perera R, Ryan M, Deeks J, Cook J. Surveillance for ocular hypertension : an evidence synthesis and economic evaluation. Health Technol Assess. 2012;16(29):1–272. source
Not checked in the free run.
- cites [10] Robertson C, Arcot Ragupathy SK, Boachie C, Dixon JM, Fraser C, Hernández R, et al. The clinical effectiveness and cost-effectiveness of different surveillance mammography regimens after the treatment for primary breast cancer: systematic reviews registry database analyses and economic evaluation. Health Technol Assess (Winch. Eng.). 2011;15(34):v–322.
Not checked in the free run.
- cites [34] Hernández RA, Jenkinson D, Vale L, Cuthbertson BH. Economic evaluation of nurse-led intensive care follow-up programmes compared with standard care: the PRaCTICaL trial. Eur J Health Econ. 2014;15(3):243–52. source
Not checked in the free run.
- cites [16] Dawes HA, Docherty T, Traynor I, Gilmore DH, Jardine AG, Knill-Jones R. Specialist nurse supported discharge in gynaecology: a randomised comparison and economic evaluation. Eur J Obstet Gynecol Reprod Biol. 2007;130(2):262–70. source
Not checked in the free run.
- cites [35] Karnon J, Partington A, Horsfall M, Chew D. Variation in clinical practice: a priority setting approach to the staged funding of quality improvement. Appl Health Econ Health Policy. 2016;14(1):21–7. source
Not checked in the free run.
- Also stated in: § Results · plain
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataReasons for exclusion of articles were: no implementation or quality improvement research aspects (575); no economic aspects (447); not within a health care/public health setting (437); language other than English (22); incomplete (19); or not a full refereed publication (37).
“Reasons for exclusion were as follows: the study did not include implementation or quality improvement research aspects (575); it did not include economic aspects (447); was not within a health care/public health setting (437); it was in a language other than English (22); it was incomplete (19); or it was not a full refereed publication (e.g. conference abstracts, doctoral theses) (37).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThree of the included studies were literature reviews.
“Three of the included studies were literature reviews [11, 13, 36].”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [11] Umscheid CA, Williams K, Brennan PJ. Hospital-based comparative effectiveness centers: translating research into practice to improve the quality, safety and value of patient care. J Gen Intern Med. 2010;25(12):1352–5. source
Not checked in the free run.
- cites [13] Faulkner A, Mills N, Bainton D, Baxter K, Kinnersley P, Peters TJ. Sharp D. a systematic review of the effect of primary care-based service innovations on quality and patterns of referral to specialist secondary care. Br J Gen Pract. 2003;53(496):878–84.
Not checked in the free run.
- cites [36] Mdege ND, Chindove S, Ali S. The effectiveness and cost implications of task-shifting in the delivery of antiretroviral therapy to HIV-infected patients: a systematic review. Health Policy Plan. 2012;28(3):223–36. source
Not checked in the free run.
May combine two claims.
Findingown dataNineteen studies were ex-post economic evaluations; of those, 17 were cost-effectiveness analyses.
“Nineteen studies were ex-post economic evaluations of which 17 were CEAs with one CUA [7, 12, 14, 15, 17] [18–30, 33].”
- Also stated in: § Results · plain
“Most studies (N = 19) were ex-post economic evaluations carried out empirically—of those, 17 were cost effectiveness analyses.”
Quote in the paperNot a substantial claim0 of 10 sources checked- cites [7] Latour CHM, Bosmans JE, van Tulder MW, de Vos R, Huyse FJ, de Jonge P, et al. Cost-effectiveness of a nurse-led case management intervention in general medical outpatients compared with usual care: an economic evaluation alongside a randomized controlled trial. J Psychosom Res. 2007;62(3):363–70. source
Not checked in the free run.
- cites [12] Albers-Heitner CP, Joore MA, Winkens RA, Lagro-Janssen AL, Severens JL, Berghmans LC. Cost-effectiveness of involving nurse specialists for adult patients with urinary incontinence in primary care compared to care-as-usual: an economic evaluation alongside a pragmatic randomized controlled trial. Neurourol Urodyn. 2012;31(4):526–34. source
Not checked in the free run.
- cites [14] Bauer JC. Nurse practitioners as an underutilized resource for health reform: evidence-based demonstrations of cost-effectiveness. J Am Acad Nurse Pract. 2010;22(4):228–31. source
Not checked in the free run.
- cites [15] Brunenberg DE, van Steyn MJ, Sluimer JC, Bekebrede LL, Bulstra SK, Joore MA. Joint recovery programme versus usual care: an economic evaluation of a clinical pathway for joint replacement surgery. Med Care. 2005:1018–26. source
Not checked in the free run.
- cites [17] Furze G, Cox H, Morton V, Chuang LH, Lewin RJ, Nelson P, Carty R, Norris H, Patel N, Elton P. Randomized controlled trial of a lay-facilitated angina management programme. J Adv Nurs. 2012;68(10):2267–79. source
Not checked in the free run.
- cites [18] Judd WR, Stephens DM, Kennedy CA. Clinical and economic impact of a quality improvement initiative to enhance early recognition and treatment of sepsis. Ann Pharmacother. 2014;48(10):1269–75. source
Not checked in the free run.
- cites [19] Kifle YA, Nigatu TH. Cost-effectiveness analysis of clinical specialist outreach as compared to referral system in Ethiopia: an economic evaluation. Cost Eff Resource Allocation. 2010;8(1):13. source
Not checked in the free run.
- cites [20] Kilpatrick K, Kaasalainen S, Donald F, Reid K, Carter N, Bryant-Lukosius D, Martin-Misener R, Harbman P, Marshall DA, Charbonneau-Smith R, DiCenso A. The effectiveness and cost-effectiveness of clinical nurse specialists in outpatient roles: a systematic review. J Eval Clin Pract. 2014;20(6):1106–23. source
Not checked in the free run.
- cites [21] Maloney S, Haas R, Keating JL, Molloy E, Jolly B, Sims J, Morgan P, Haines T. Breakeven, cost benefit, cost effectiveness, and willingness to pay for web-based versus face-to-face education delivery for health professionals. J Med Internet Res. 2012;14(2):e47. source
Not checked in the free run.
- cites [22] Mortimer D, French SD, McKenzie JE, Denise AO, Green SE. Economic evaluation of active implementation versus guideline dissemination for evidence-based care of acute low-back pain in a general practice setting. PLoS One. 2013;8(10):e75647. source
Not checked in the free run.
- Also stated in: § Results · plain
May combine two claims.
Findingown dataAll of the studies included in the review were empirical and ex-post in nature.
“all of the studies included in our review were empirical and ex-post in nature.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataStudy quality was diminished through use of post-hoc subgroup analysis, limited reporting of the handling of uncertainty and justification for choice of discount rates.
“diminished through use of post-hoc subgroup analysis, limited reporting of the handling of uncertainty and justification for choice of discount rates.”
Quote in the paperNot a substantial claimMay combine two claims.
FindingreasoningThe criteria used in this review are stricter than those applied to prior reviews, which were more inclusive of qualitative outcomes and costing studies.
“Taken together, these criteria are stricter than those applied to prior reviews, which were more inclusive of qualitative outcomes and costing studies.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataScenario analysis for rollout or scaling up was included in three of the studies, and potential funding sources were considered by one study.
“Scenario analysis for rollout or scaling up was included in three of the studies, and potential funding sources were considered by one study.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataJustification for the outcome measures used was not reported in any of the studies included in Hoomans et al. but reported in 68% of studies included here.
“Justification for the outcome measures used was not reported in any of the studies included in Hoomans et al. but reported in 68% of studies included here.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Hoomans et al.
Not checked in the free run.
May combine two claims.
Findingown dataOnly 42% of studies reviewed by Hoomans et al. included evaluation of costs and outcomes against 'standard practice/status quo' comparators, compared to 95% of studies in this review.
“only 42% of studies reviewed by Hoomans et al. included evaluation of costs and outcomes against ‘standard practice/status quo’ comparators, compared to 95% of studies in our review.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Hoomans et al.
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductnone givenThe databases were chosen to attempt to capture the widest range of health improvement, social scientific and health economic studies.
“These databases were chosen to attempt to capture the widest range of health improvement, social scientific and health economic studies.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe reference lists of the included studies were checked for any related studies not picked up by the search.
“The reference lists of the studies were checked for any related studies that were not picked up by the search.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe QHES instrument was designed to more easily tell the difference between high-quality and low-quality studies.
“The QHES instrument was designed to more easily tell the difference between high-quality and low-quality studies [5].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [5] Ofman JJ, Sullivan SD, Neumann PJ, Chiou C-F, Henning JM, Wade SW, et al. Examining the value and quality of health economic analyses: implications of utilizing the QHES. J Manag Care Pharm. 2003;9(1):53–61. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationEach paper's methodological quality was assessed using the Quality of Health Economic Studies (QHES) standardised framework.
“Each paper’s methodological quality was assessed using the Quality of Health Economic Studies (QHES) standardised framework [4].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [4] Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Int J Technol Assess Health Care. 2005;21(1):149. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningTo mitigate potential selection bias after screening, keyword searching was done in Rayyan for main keywords within excluded categories, and these were re-screened by the first author.
“To mitigate for potential selection bias after screening, keyword searching was done in Rayyan for the main keywords within the excluded categories (primarily, those that were deemed to be topic-relevant but not containing economic methods). These were then re-screened by the first author.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenA systematic review methodology was undertaken.
“A systematic review methodology was undertaken.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenData extracted during screening were cross-checked for accuracy during the data extraction stage.
“These were cross-checked for accuracy during the data extraction stage.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationAs per standard practice using QHES, studies were deemed good quality if they attained a score of 75/100 or higher.
“As per standard practice using this framework, the studies were deemed to be of good quality if they attained a score of 75/100 or higher [5].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [5] Ofman JJ, Sullivan SD, Neumann PJ, Chiou C-F, Henning JM, Wade SW, et al. Examining the value and quality of health economic analyses: implications of utilizing the QHES. J Manag Care Pharm. 2003;9(1):53–61. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe review only considered papers reported in English.
“We also only considered papers reported in English.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
InterpretationreasoningThere remains scope for continued improvement in key areas and for increased collaboration between health economics and implementation science.
“though there remains scope for continued improvement in key areas and for increased collaboration between health economics and implementation science.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningThe strict inclusion criteria mean that the reviewed evidence is cohesive.
“The strict inclusion criteria mean that the reviewed evidence is cohesive.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningThe results of evaluating the 30 included papers paint a picture of an area of research that is still developing.
“The results of evaluating the 30 included papers paint a picture of an area of research that is still developing.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductreasoningThe systematic appraisal carried out allows a longitudinal critique of the quality of economic studies in this field.
“The systematic appraisal we carried out also allows us a longitudinal critique of the quality of economic studies in this field.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationExisting implementation frameworks make reference to the need to consider costs as part of an evaluative research strategy, but do not specify how this is to be done.
“Existing implementation frameworks [40] make reference to the need to consider costs as part of an evaluative research strategy, but do not specify how this is to be done.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [40] Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38(2):65–76. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationcitationThe QHES used here is based on Drummond's guidelines and is designed to cover the same topics, but offers a simpler, quantifiable format that is easier to apply.
“we would argue that the QHES used here is based on Drummond’s guidelines (used in prior reviews) and is designed to cover the same topics, but offers a simpler, quantifiable format that is easier to apply. [32]”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [32] Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, Augustovski F, Briggs AH, Mauskopf J, Loder E. Consolidated health economic evaluation reporting standards (CHEERS) statement. Cost Eff Resource Allocation. 2013;11(1):6. source
Not checked in the free run.
The quote only partly states this claim.
Interpretationown dataThe quality of published economic evaluations in the field is overall good.
“however, quality is overall good.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
InterpretationreasoningThe expanding application of health economic evaluations within the improvement and implementation sphere where high-quality study reporting has been a major recent focus has also plausibly contributed to improved reporting.
“Additionally, the expanding application of health economic evaluations within the improvement and implementation sphere where high-quality study reporting has been a major recent focus has also plausibly contributed to improved reporting.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
The relationship between implementation outcomes, service outcomes and patient outcomes is central to understanding the benefits and costs and overall cost-effectiveness of an intervention.
“The relationship between implementation outcomes, service outcomes and patient outcomes is central to understanding the benefits and costs and overall cost-effectiveness of an intervention.”
The paper relies on this without support.
Economic evaluation can inform choices over whether and how resources should be allocated to improve services and for implementing evidence into health care practice.
“Economic evaluation can inform choices over whether and how resources should be allocated to improve services and for implementing evidence into health care practice.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Implementation researchers are increasingly using economic evaluation to explore the benefits produced by implementing evidence-based practices (EBPs) in healthcare settings.
“Implementation researchers are increasingly using economic evaluation to explore the benefits produced by implementing evidence-based practices (EBPs) in healthcare settings.”
From Translating economic evaluations into financing strategies for implementing evidence-based practices (2021) · cites this paper as [3]
Almost all implementation research involving financing strategies or economic evaluations has focused on active implementation.
“Almost all implementation research involving financing strategies [24] or economic evaluations [2, 3] has focused on active implementation, yet all phases are relevant to financing decisions.”
From Translating economic evaluations into financing strategies for implementing evidence-based practices (2021) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Discussion calls all studies empirical despite reported modelling studies
MajorThe results identify four cost-utility analyses using modelling rather than empirical methods, contradicting the discussion's claim that all included studies were empirical.
“There were also four further CUAs that used economic modelling rather than empirical methods [8–10, 34], and two cost-consequence analyses [16, 35].”
“we found that all of the studies included in our review were empirical and ex-post in nature.”
Search and exclusion totals leave 72 articles unaccounted for
MajorSubtracting the 1566 excluded articles from the 1668 identified articles leaves 102, not the 30 reported as included. The flow described in the text does not account for the other 72 articles.
Reported 30 · recomputed 102
“In total, the initial search strategy identified 1668 articles, of which 1566 were excluded, 1525 during the initial screen and 41 following full text screening.”
Reasons for exclusion do not sum to the exclusion total
MinorThe six reported exclusion categories total 1537 articles, 29 fewer than the stated 1566 exclusions.
Reported 1566 · recomputed 1537
“Reasons for exclusion were as follows: the study did not include implementation or quality improvement research aspects (575); it did not include economic aspects (447); was not within a health care/public health setting (437); it was in a language other than English (22); it was incomplete (19); or it was not a full refereed publication (e.g. conference abstracts, doctoral theses) (37).”
European studies are not 62% of the reviewed studies
MinorSeventeen of the 30 included studies is 56.7%, not 62%.
Reported 62 · recomputed 56.666667
“Seventeen of the studies (62%) were European-based”
Six North American studies are not 23% of the reviewed studies
MinorSix of the 30 included studies is 20%, not 23%.
Reported 23 · recomputed 20
“six studies (23%) were based in either the USA or Canada”
Reported proportion of good-quality studies does not match its count
MinorThe paper says 22 papers underwent QHES appraisal. Thirteen of 22 is 59.1%, not the reported 62%.
Reported 62 · recomputed 59.090909
“Thirteen of the studies (62%) attained a ‘good’ score of over 75.”
A study dated 2003 appears despite the post-2003 eligibility rule
MinorThe included-studies table lists a 2003 study, although the eligibility criteria require publication after 2003.
“| Faulkener et al | 2003 | UK | Improvement | Primary Care | Improved referral | N/A | Review | Review | N/A |”
“Were published after 2003 and before 16th March 2016”
Rollout or scaling scenarios appear in more than three studies
MinorThe text reports three studies with rollout or scaling scenarios, but Table 5 lists rollout scenarios for Maloney, Mdege, Mortimer and Purshouse, as well as scaling scenarios for Judd.
“Scenario analysis for rollout or scaling up was included in three of the studies”
“| Maloney et al. 2012 | Training and set up costs | Roll out scenarios | Yes | | Mdege et al. 2012 | Training costs | Roll out scenarios | Yes | | Mortimer et al. 2013 | Development costs; amortisation; delivery costs; roll out costs | Roll out scenarios | No | | Purshouse et al. 2013 | None | Roll out scenarios | Yes, although sensitive to rollout costs |”
Cites
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · see citing works
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015) · Implementation Science · doi:10.1186/s13012-015-0209-1 · Cited by 4,963 · see citing works
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012) · Medical Care · doi:10.1097/mlr.0b013e3182408812 · Cited by 4,058 · see citing works
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., Vale, L., Whitty, P., Eccles, M., Matowe, L., Shirran, L., et al. (2004) · Health Technology Assessment · doi:10.3310/hta8060 · Cited by 3,044 · see citing works
Guidelines for authors and peer reviewers of economic submissions to the BMJ
Drummond, M. F., & Jefferson, T. O. (1996) · BMJ · doi:10.1136/bmj.313.7052.275 · Cited by 1,827 · see citing works
What is "quality improvement" and how can it transform healthcare?
Batalden, P. B., & Davidoff, F. (2007) · BMJ Quality & Safety · doi:10.1136/qshc.2006.022046 · Cited by 1,104 · see citing works
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J. M., Thomas, R. E., MacLennan, G., Fraser, C., Ramsay, C. R., Vale, L., Whitty, P., Eccles, M. P., Matowe, L., Shirran, L., et al. (2005) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462305290190 · Cited by 864 · see citing works
Burr, J. M., Mowatt, G., Hernández, R. A., Siddiqui, M. A. R., Cook, J. A., Lourenço, T., Ramsay, C. R., Vale, L. D., Fraser, C., Azuara‐Blanco, A., et al. (2007) · Health Technology Assessment · doi:10.3310/hta11410 · Cited by 549 · see citing works
An Overview of Research and Evaluation Designs for Dissemination and Implementation
Brown, C. H., Curran, G., Palinkas, L. A., Aarons, G. A., Wells, K. B., Jones, L., Collins, L. M., Duan, N., Mittman, B. S., Wallace, A., et al. (2017) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044215 · Cited by 509 · see citing works
Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement
Husereau, D., Drummond, M., Petrou, S., Carswell, C., Moher, D., Greenberg, D., Augustovski, F., Briggs, A. H., Mauskopf, J., Loder, E., et al. (2013) · Cost Effectiveness and Resource Allocation · doi:10.1186/1478-7547-11-6 · Cited by 381 · see citing works
Examining the Value and Quality of Health Economic Analyses: Implications of Utilizing the QHES
Ofman, J. J., Sullivan, S. D., Neumann, P. J., Chiou, C.-F., Henning, J. M., Wade, S. W., & Hay, J. W. (2003) · Journal of Managed Care Pharmacy · doi:10.18553/jmcp.2003.9.1.53 · Cited by 366 · see citing works
Economic evaluation of implementation strategies in health care
Hoomans, T., & Severens, J. L. (2014) · Implementation Science · doi:10.1186/s13012-014-0168-y · Cited by 181 · see citing works
Surveillance for ocular hypertension: an evidence synthesis and economic evaluation.
Burr, J., Botello-Pinzon, P., Takwoingi, Y., Hernández, R., Vazquez-Montes, M., Elders, A., Asaoka, R., Banister, K., van der Schoot, J., Fraser, C., et al. (2012) · Health Technology Assessment · doi:10.3310/hta16290 · Cited by 180 · see citing works
Systematic review and evaluation of methods of assessing urinary incontinence
Martin, J., Williams, K., Abrams, K. R., Turner, D., Sutton, A. J., Chapple, C. R., Assassa, R. P., Shaw, C., & Cheater, F. (2006) · Health Technology Assessment · doi:10.3310/hta10060 · Cited by 164 · see citing works
Bauer, J. C. (2010) · Journal of the American Academy of Nurse Practitioners · doi:10.1111/j.1745-7599.2010.00498.x · Cited by 147 · see citing works
Mdege, N. D., Chindove, S., & Ali, S. (2013) · Health Policy and Planning · doi:10.1093/heapol/czs058 · Cited by 137 · see citing works
Tappenden, P., Campbell, F., Rawdin, A. C., Wong, R., & Kalita, N. (2012) · Health Technology Assessment · doi:10.3310/hta16200 · Cited by 135 · see citing works
Robertson, C., Ragupathy, S. A., Boachie, C., Dixon, J. M., Fraser, C., Hernández, R. A., Heys, S. D., Jack, W., Kerr, G., Lawrence, G., et al. (2011) · Health Technology Assessment · doi:10.3310/hta15340 · Cited by 113 · see citing works
Brunenberg, D. E., van Steyn, M. J., Sluimer, J. C., Bekebrede, L. L., Bulstra, S. K., & Joore, M. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000178266.75744.35 · Cited by 108 · see citing works
Maloney, S., Haas, R., Keating, J. L., Molloy, E., Jolly, B., Sims, J., Morgan, P., & Haines, T. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2040 · Cited by 107 · see citing works
Kilpatrick, K., Kaasalainen, S., Donald, F., Reid, K., Carter, N., Bryant‐Lukosius, D., Martin‐Misener, R., Harbman, P., Marshall, D. A., Charbonneau‐Smith, R., et al. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12219 · Cited by 98 · see citing works
Purshouse, R. C., Brennan, A., Rafia, R., Latimer, N. R., Archer, R. J., Angus, C. R., Preston, L. R., & Meier, P. S. (2013) · Alcohol and Alcoholism · doi:10.1093/alcalc/ags103 · Cited by 83 · see citing works
Faulkner, A., Mills, N., Bainton, D., Baxter, K., Kinnersley, P., Peters, T. J., & Sharp, D. (2003) · Europe PMC (PubMed Central) · Cited by 79 · see citing works
Hoomans, T., Evers, S. M. A. A., Ament, A. J. H. A., Hübben, M. W. A., van der Weijden, T., Grimshaw, J. M., & Severens, J. L. (2007) · Value in Health · doi:10.1111/j.1524-4733.2007.00175.x · Cited by 71 · see citing works
Williams, K. S., Assassa, R. P., Cooper, N. J., Turner, D. A., Shaw, C., Abrams, K. R., Mayne, C., Jagger, C., Matthews, R. J., Clarke, M., et al. (2005) · PubMed · Cited by 68 · see citing works
Williams, J. G., Russell, I. T., Durai, D., Cheung, W., Farrin, A. J., Bloor, K. E., Coulton, S., & Richardson, G. A. (2006) · Health Technology Assessment · doi:10.3310/hta10400 · Cited by 48 · see citing works
Umscheid, C. A., Williams, K., & Brennan, P. J. (2010) · Journal of General Internal Medicine · doi:10.1007/s11606-010-1476-9 · Cited by 41 · see citing works
Judd, W. R., Stephens, D. M., & Kennedy, C. A. (2014) · Annals of Pharmacotherapy · doi:10.1177/1060028014541792 · Cited by 35 · see citing works
Albers‐Heitner, C. P., Joore, M. A., Winkens, R. A. G., Lagro‐Janssen, A. L. M., Severens, J. L., & Berghmans, L. C. M. (2012) · Neurourology and Urodynamics · doi:10.1002/nau.21204 · Cited by 33 · cited 3× · see citing works
“Cost-utility analysis (CUA): compares costs and outcomes of alternatives with outcomes measured as quality-adjusted life years (QALYs) gained.”
BackgroundInfluentialSpecialist nurse supported discharge in gynaecology: A randomised comparison and economic evaluation
Dawes, H. A., Docherty, T., Traynor, I., Gilmore, D. H., Jardine, A. G., & Knill-Jones, R. (2007) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2006.02.002 · Cited by 32 · see citing works
Randomized controlled trial of a lay‐facilitated angina management programme
Furze, G., Cox, H., Morton, V., Chuang, L., Lewin, R. J. P., Nelson, P., Carty, R., Norris, H., Patel, N., & Elton, P. (2012) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2011.05920.x · Cited by 31 · see citing works
Mortimer, D., French, S. D., McKenzie, J. E., O′Connor, D. A., & Green, S. E. (2013) · PLoS ONE · doi:10.1371/journal.pone.0075647 · Cited by 30 · see citing works
Hernández, R. A., Jenkinson, D., Vale, L., & Cuthbertson, B. H. (2014) · The European Journal of Health Economics · doi:10.1007/s10198-013-0470-7 · Cited by 28 · see citing works
Evidence‐based care pathway for cellulitis improves process, clinical, and cost outcomes
Yarbrough, P. M., Kukhareva, P., Spivak, E. S., Hopkins, C., & Kawamoto, K. (2015) · Journal of Hospital Medicine · doi:10.1002/jhm.2433 · Cited by 27 · see citing works
Tappenden, P., Chilcott, J. B., Brennan, A., Squires, H., Glynne‐Jones, R., & Tappenden, J. (2013) · Value in Health · doi:10.1016/j.jval.2013.02.012 · Cited by 26 · see citing works
Latour, C. H. M., Bosmans, J. E., van Tulder, M. W., de Vos, R., Huyse, F. J., de Jonge, P., van Gemert, L. A. M., & Stalman, W. A. B. (2007) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2006.10.016 · Cited by 17 · see citing works
Karnon, J., Partington, A., Horsfall, M., & Chew, D. (2016) · Applied Health Economics and Health Policy · doi:10.1007/s40258-015-0160-y · Cited by 15 · see citing works
Vestergaard, A. S., & Ehlers, L. H. (2015) · PharmacoEconomics · doi:10.1007/s40273-015-0281-z · Cited by 15 · see citing works
Kifle, Y. A., & Nigatu, T. H. (2010) · Cost Effectiveness and Resource Allocation · doi:10.1186/1478-7547-8-13 · Cited by 13 · see citing works
Haji Ali Afzali, H., Gray, J., Beilby, J., Holton, C., & Karnon, J. (2013) · Applied Health Economics and Health Policy · doi:10.1007/s40258-013-0062-9 · Cited by 10 · see citing works
The economics of health service transformation
Rachev, B. T. (2015) · Clinical Governance An International Journal · doi:10.1108/cgij-07-2015-0024 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Elmagarmid A, Fedorowicz Z, Hammady H, Ilyas I, Khabsa M, Ouzzani M. Rayyan: a systematic reviews web app for exploring and filtering searches for eligible studies for Cochrane reviews. In: Evidence-informed public health: opportunities and challenges. Abstracts of the 22nd Cochrane colloquium. Hyderabad: Wiley; 2014. p. 21–6.
Cited by
Sorted by citations
Beidas, R. S., Dorsey, S., Lewis, C. C., Lyon, A. R., Powell, B. J., Purtle, J., Saldana, L., Shelton, R. C., Stirman, S. W., & Lane-Fall, M. B. (2022) · Implementation Science · doi:10.1186/s13012-022-01226-3 · Cited by 230 · see citing works
A pragmatic method for costing implementation strategies using time-driven activity-based costing
Cidav, Z., Mandell, D., Pyne, J., Beidas, R., Curran, G., & Marcus, S. (2020) · Implementation Science · doi:10.1186/s13012-020-00993-1 · Cited by 174 · see citing works
Cost data in implementation science: categories and approaches to costing
Gold, H. T., McDermott, C., Hoomans, T., & Wagner, T. H. (2022) · Implementation Science · doi:10.1186/s13012-021-01172-6 · Cited by 152 · see citing works
Sohn, H., Tucker, A., Ferguson, O., Gomes, I., & Dowdy, D. W. (2020) · Implementation Science · doi:10.1186/s13012-020-01047-2 · Cited by 116 · see citing works
A Review of Interventions to Enhance HIV Medication Adherence
Whiteley, L., Olsen, E. M., Haubrick, K. K., Odoom, E., Tarantino, N., & Brown, L. K. (2021) · Current HIV/AIDS Reports · doi:10.1007/s11904-021-00568-9 · Cited by 95 · see citing works
de la Perrelle, L., Radisic, G., Cations, M., Kaambwa, B., Barbery, G., & Laver, K. (2020) · BMC Health Services Research · doi:10.1186/s12913-020-4981-5 · Cited by 90 · see citing works
Aggarwal, N. K., Jarvis, G. E., Gómez-Carrillo, A., Kirmayer, L. J., & Lewis‐Fernández, R. (2020) · Transcultural Psychiatry · doi:10.1177/1363461520940481 · Cited by 68 · see citing works
Bowser, D. M., Henry, B. F., & McCollister, K. E. (2021) · Implementation Science · doi:10.1186/s13012-021-01094-3 · Cited by 61 · see citing works
The cost of oral cancer: A systematic review
Ribeiro‐Rotta, R. F., Rosa, E. A., Milani, V., Dias, N. R., Masterson, D., da Silva, E. N., & Zara, A. L. D. S. A. (2022) · PLoS ONE · doi:10.1371/journal.pone.0266346 · Cited by 59 · see citing works
Extending analytic methods for economic evaluation in implementation science
O’Leary, M. C., Lich, K. H., Frerichs, L., Leeman, J., Reuland, D. S., & Wheeler, S. B. (2022) · Implementation Science · doi:10.1186/s13012-022-01192-w · Cited by 56 · see citing works
Wolcherink, M. O. O., Behr, C., Pouwels, X., Doggen, C., & Koffijberg, H. (2023) · PharmacoEconomics (Auckland) · doi:10.1007/s40273-023-01287-2 · Cited by 54 · see citing works
Translating economic evaluations into financing strategies for implementing evidence-based practices
Dopp, A. R., Kerns, S. E. U., Panattoni, L., Ringel, J. S., Eisenberg, D., Powell, B. J., Low, R., & Raghavan, R. (2021) · Implementation Science · doi:10.1186/s13012-021-01137-9 · Cited by 52 · see citing works
Wilson, P. M., & Kislov, R. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009237055 · Cited by 52 · see citing works
A guide to systems-level, participatory, theory-informed implementation research in global health
Seward, N., Hanlon, C., Hinrichs, S., Lund, C., Murdoch, J., Salisbury, T. T., Verhey, R., Shidhaye, R. R., Thornicroft, G., Araya, R., et al. (2021) · BMJ Global Health · doi:10.1136/bmjgh-2021-005365 · Cited by 51 · see citing works
Self-citationXu, R. H., Keetharuth, A. D., Wang, L.-L., Cheung, A. W.-L., & Wong, E. L. (2021) · The European Journal of Health Economics · doi:10.1007/s10198-021-01359-0 · Cited by 51 · see citing works
Implementation costs of hospital-based computerised decision support systems: a systematic review
Donovan, T., Abell, B., Fernando, M., McPhail, S., & Carter, H. E. (2023) · Implementation Science · doi:10.1186/s13012-023-01261-8 · Cited by 44 · see citing works
Krebs, E., & Nosyk, B. (2021) · Current HIV/AIDS Reports · doi:10.1007/s11904-021-00550-5 · Cited by 39 · see citing works
Collaborating with health economists to advance implementation science: a qualitative study
Barnett, M. L., Dopp, A. R., Klein, C. C., Ettner, S. L., Powell, B. J., & Saldana, L. (2020) · Implementation Science Communications · doi:10.1186/s43058-020-00074-w · Cited by 38 · see citing works
Salvati, Z. M., Rahm, A. K., Williams, M. S., Ladd, I., Schlieder, V., Atondo, J., Schneider, J. L., Epstein, M. M., Lu, C. Y., Pawloski, P. A., et al. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00424-4 · Cited by 34 · see citing works
Malhotra, A., Thompson, R. R., Kagoya, F., Masiye, F., Mbewe, P., Mosepele, M., Phiri, J., Sambo, J., Barker, A. R., Cameron, D., et al. (2022) · Implementation Science · doi:10.1186/s13012-022-01248-x · Cited by 33 · see citing works
Yeung, K., Richards, J. A., Goemer, E., Lozano, P., Lapham, G. T., Williams, E. C., Glass, J. E., Lee, A., Achtmeyer, C. E., Caldeiro, R. M., et al. (2020) · Health Services Research · doi:10.1111/1475-6773.13592 · Cited by 32 · see citing works
Michaud, T. L., Pereira, E. L., Porter, G. C., Golden, C. A., Hill, J. L., Kim, J., Wang, H., Schmidt, C., & Estabrooks, P. A. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-060785 · Cited by 24 · see citing works
Rethinking How We Measure Costs in Implementation Research
Wagner, T. H. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-06104-6 · Cited by 23 · see citing works
Using decision analysis to support implementation planning in research and practice
Smith, N., Knocke, K., & Lich, K. H. (2022) · Implementation Science Communications · doi:10.1186/s43058-022-00330-1 · Cited by 22 · see citing works
Vaughn, A., Studts, C. R., Powell, B. J., Ammerman, A. S., Trogdon, J. G., Curran, G. M., Hales, D. P., Willis, E. A., & Ward, D. S. (2019) · Implementation Science · doi:10.1186/s13012-019-0949-4 · Cited by 21 · see citing works
Harden, S. M., Balis, L. E., Strayer, T. E., & Wilson, M. L. (2021) · Preventing Chronic Disease · doi:10.5888/pcd18.200513 · Cited by 21 · see citing works
“Cost‐effectiveness of ACL treatment is dependent on age and activity level: a systematic review”
Deviandri, R., van der Veen, H. C., Lubis, A. M. T., van den Akker‐Scheek, I., & Postma, M. J. (2022) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-022-07087-z · Cited by 20 · see citing works
Nguyen, M. X. B., Chu, A. V., Powell, B. J., Tran, H. V., Nguyen, L. H., Dao, A. T. M., Pham, M. D., Vo, S. H., Bui, N. H., Dowdy, D. W., et al. (2020) · Implementation Science · doi:10.1186/s13012-020-01020-z · Cited by 19 · see citing works
Huebschmann, A. G., Glasgow, R. E., Leavitt, I. M., Chapman, K., Rice, J. D., Lockhart, S., Stevens‐Lapsley, J. E., Reusch, J. E. B., Dunn, A. L., & Regensteiner, J. G. (2022) · Translational Behavioral Medicine · doi:10.1093/tbm/ibac014 · Cited by 17 · see citing works
Thusini, S., Milenova, M., Nahabedian, N., Grey, B., Soukup, T., & Henderson, C. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08171-3 · Cited by 17 · see citing works
Cidav, Z., Mandell, D. S., Ingersoll, B. R., & Pellecchia, M. (2023) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-022-01247-6 · Cited by 17 · see citing works
Scholl, I., Kobrin, S. C., & Elwyn, G. J. (2020) · Implementation Science · doi:10.1186/s13012-020-01042-7 · Cited by 17 · see citing works
Szewczyk, Z., Reeves, P., Kingsland, M., Doherty, E. L., Elliott, E. J., Wolfenden, L., Tsang, T. W., Dunlop, A. J., Searles, A., & Wiggers, J. H. (2022) · Implementation Science · doi:10.1186/s13012-021-01180-6 · Cited by 16 · see citing works
The economics of adaptations to evidence-based practices
Salloum, R. G., Wagner, T. H., Midboe, A. M., Daniels, S. I., Quanbeck, A., & Chambers, D. (2022) · Implementation Science Communications · doi:10.1186/s43058-022-00345-8 · Cited by 16 · see citing works
Hartzler, B. J., Hinde, J. M., Lang, S. G., Correia, N., Yermash, J., Yap, K., Murphy, C. M., Ruwala, R., Rash, C. J., Becker, S. J., et al. (2022) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-022-00283-1 · Cited by 16 · see citing works
Obesity prevention practices in early care and education settings: an adaptive implementation trial
Swindle, T., Rutledge, J. M., Selig, J. P., Painter, J. T., Zhang, D., Martin, J., Johnson, S. L., Whiteside-Mansell, L., Almirall, D., McElwee, T. M. B., et al. (2022) · Implementation Science · doi:10.1186/s13012-021-01185-1 · Cited by 15 · see citing works
Huebschmann, A. G., Trinkley, K. E., Gritz, M., & Glasgow, R. E. (2022) · Implementation Science Communications · doi:10.1186/s43058-022-00292-4 · Cited by 15 · see citing works
Belay, Y. B., Engel, L., Lee, Y. Y., Le, N., & Mihalopoulos, C. (2023) · PharmacoEconomics · doi:10.1007/s40273-023-01257-8 · Cited by 15 · see citing works
Yeshitila, Y. G., Gold, L. H., Abimanyi-Ochom, J., Riggs, E., Tolossa, T., & Le, H. (2024) · Social Science & Medicine · doi:10.1016/j.socscimed.2024.117250 · Cited by 14 · see citing works
Sáenz‐Ravello, G., Castillo‐Riquelme, M., Cuadrado, C., Gamonal, J. A., & Baeza, M. (2024) · Community Dentistry And Oral Epidemiology · doi:10.1111/cdoe.12999 · Cited by 14 · see citing works
Donovan, T., Abell, B., McPhail, S., & Carter, H. E. (2025) · Implementation Science · doi:10.1186/s13012-025-01423-w · Cited by 14 · see citing works
Bartáková, J., Zúñiga, F., Guerbaaï, R.-A., Basinska, K., Brunkert, T., Simon, M., Denhaerynck, K., Geest, S. D., Wellens, N. I. H., Serdaly, C., et al. (2022) · BMC Geriatrics · doi:10.1186/s12877-022-03182-5 · Cited by 14 · see citing works
Barnett, M. L., Stadnick, N. A., Proctor, E. K., Dopp, A. R., & Saldana, L. (2021) · Implementation Science Communications · doi:10.1186/s43058-021-00239-1 · Cited by 13 · see citing works
Karstens, S., Lang, S., & Saunders, B. (2020) · Health Services Insights · doi:10.1177/1178632920977894 · Cited by 13 · see citing works
Munishi, C. G., Mateshi, G., Mlunde, L. B., Njiro, B. J., Ngowi, J. E., Kengia, J. T., Kapologwe, N., Deng, L., Timbrell, A., Kitinya, W. J., et al. (2023) · PLOS Global Public Health · doi:10.1371/journal.pgph.0001487 · Cited by 12 · see citing works
Ahern, S., Riordan, F., Murphy, A., Browne, J. P., Kearney, P. M., Smith, S. M., & McHugh, S. (2021) · Implementation Science · doi:10.1186/s13012-021-01085-4 · Cited by 12 · see citing works
Reporting standards, outcomes and costs of quality improvement studies in Ireland: a scoping review
McCarthy, S. E., Jabakhanji, S. B., Martin, J., Flynn, M. A., & Sørensen, J. R. (2021) · BMJ Open Quality · doi:10.1136/bmjoq-2020-001319 · Cited by 12 · see citing works
Nelson, R. E., Goto, M., Samore, M. H., Jones, M., Stevens, V. W., Evans, M. E., Schweizer, M. L., Perencevich, E. N., & Rubin, M. A. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa1591 · Cited by 12 · see citing works
Jorm, C. M., Iedema, R. A. M., Piper, D. A., Goodwin, N., & Searles, A. (2021) · Journal of Health Organization and Management · doi:10.1108/jhom-06-2020-0218 · Cited by 12 · see citing works
Rejon, C. S., Hollingworth, W., Opmeer, B. C., Luyt, K., & McLeod, H. (2025) · Applied Health Economics and Health Policy · doi:10.1007/s40258-025-00993-6 · Cited by 11 · see citing works
Lehman, V. E. R., Siegel, J. E., & Chiang, E. N. (2023) · Medical Care · doi:10.1097/mlr.0000000000001873 · Cited by 11 · see citing works
Kirchner, J. E., Waltz, T. J., Powell, B. J., Woodward, E. N., Smith, J. L., & Proctor, E. K. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780197660690.003.0006 · Cited by 11 · see citing works
Burns, C. L., Wishart, L. R., Kularatna, S., & Ward, E. C. (2020) · Speech Language and Hearing · doi:10.1080/2050571x.2019.1693750 · Cited by 11 · see citing works
Economic Evaluation of Implementation Strategies
Severens, J. L., Hoomans, T., Adang, E., & Wensing, M. (2020) · Wiley eBooks · doi:10.1002/9781119488620.ch23 · Cited by 11 · see citing works
Hewage, S. A., Noviyani, R., Brain, D., Sharma, P., Parsonage, W. A., McPhail, S., Barnett, A. G., & Kularatna, S. (2023) · Cost Effectiveness and Resource Allocation · doi:10.1186/s12962-023-00486-0 · Cited by 10 · see citing works
Carrandi, A., Grove, A., Skouteris, H., Melder, A., Hu, Y., Dever, M., & Higgins, A. M. (2024) · Implementation Science Communications · doi:10.1186/s43058-024-00562-3 · Cited by 10 · see citing works
Brown, V., Tran, H. N. Q., Blake, M. R., Rachel, L., & Moodie, M. (2020) · Health Research Policy and Systems · doi:10.1186/s12961-020-00633-6 · Cited by 10 · see citing works
Hinde, J. M., Garner, B. R., Watson, C. J., Ramanan, R., Ball, E. L., & Tueller, S. (2022) · Implementation Research and Practice · doi:10.1177/26334895221089266 · Cited by 10 · see citing works
Wang, Y., Guo, D., Xia, Y., Hu, M., Wang, M., Shi, Z., Guan, X., Zhu, D., & He, P. (2025) · Nature Communications · doi:10.1038/s41467-025-58120-x · Cited by 9 · see citing works
Heggie, R., Boyd, K., & Wu, O. (2021) · Health Research Policy and Systems · doi:10.1186/s12961-021-00766-2 · Cited by 9 · see citing works
Falkenbach, P., Raudasoja, A., Vernooij, R. W. M., Mustonen, J. M. J., Agarwal, A., Aoki, Y., Blanker, M. H., Cartwright, R., García‐Perdomo, H. A., Kilpeläinen, T. P., et al. (2023) · Implementation Science · doi:10.1186/s13012-023-01290-3 · Cited by 9 · see citing works
Heggie, R., Boyd, K., Kamaruzaman, H. F., & Wu, O. (2024) · Health Research Policy and Systems · doi:10.1186/s12961-024-01220-9 · Cited by 9 · see citing works
Uzoaru, F., Nwaozuru, U., Ong, J. J., Obi, F. A., Obiezu‐Umeh, C., Tucker, J. D., Shato, T., Mason, S., Carter, V., Manu, S., et al. (2021) · Implementation Science Communications · doi:10.1186/s43058-021-00177-y · Cited by 9 · see citing works
Gyllensten, H., Tistad, M., Fridberg, H., & Wallin, L. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073829 · Cited by 8 · see citing works
Food consumption and mental health in children and adolescents: A systematic review protocol
Costa, M. B., Silva, R. M. F., Silva, K. V. C., Nogueira, S. M., Terra, L. F., Cordeiro, L. L., Villa‐González, E., Noll, P. R. E. S., & Noll, M. A. (2024) · MethodsX · doi:10.1016/j.mex.2024.103015 · Cited by 8 · see citing works
Donovan, T., Carter, H. E., McPhail, S., & Abell, B. (2024) · Implementation Science Communications · doi:10.1186/s43058-024-00648-y · Cited by 8 · see citing works
Garner, B. R., Patel, S. V., & Kirk, M. A. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01535-y · Cited by 7 · see citing works
Tozan, Y., Capasso, A., Namatovu, P., Kiyingi, J., Damulira, C., Nabayinda, J., Bahar, O. S., McKay, M. M., Hoagwood, K., & Ssewamala, F. M. (2022) · American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.21-0895 · Cited by 7 · see citing works
Michaud, T. L., Hill, J. L., Heelan, K. A., Bartee, R. T., Abbey, B. M., Malmkar, A., Masker, J., Golden, C. A., Porter, G. C., Glasgow, R. E., et al. (2022) · Implementation Science Communications · doi:10.1186/s43058-022-00287-1 · Cited by 7 · see citing works
Ling, V. B., Levi, E. E., Harrington, A. R., Zite, N. B., Rivas, S. D., Dalton, V. K., Smith, R. D., & Moniz, M. H. (2020) · BMJ Quality & Safety · doi:10.1136/bmjqs-2020-011111 · Cited by 7 · see citing works
Bankole, A. O., Conombo, B., Légaré, F., & Laberge, M. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13024-w · Cited by 6 · see citing works
Implementation science on the margins: How do we demonstrate the value of implementation strategies?
Hinde, J. M., Bray, J. W., & Cowell, A. J. (2020) · Families Systems & Health · doi:10.1037/fsh0000535 · Cited by 6 · see citing works
The Role of Economic Evaluation in Dissemination and Implementation Research
Dopp, A. R., Walker, S., & Raghavan, R. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780197660690.003.0011 · Cited by 6 · see citing works
Who Conducts Health Economic Evaluations of Laboratory Tests? A Scoping Review
Giusepi, I., John, A. S., & Jülicher, P. (2020) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfaa107 · Cited by 6 · see citing works
Chen, J., Wijesundara, J. G., Patterson, A. M., Cutrona, S. L., Aiello, S., McManus, D. D., McKee, M. D., Wang, B., & Houston, T. K. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-07031-w · Cited by 6 · see citing works
Economic evaluation methods used in home‐visiting interventions: A systematic search and review
Bailey, C., Skouteris, H., Morris, H., O’Donnell, R., Hill, B., & Ademi, Z. (2021) · Health & Social Care in the Community · doi:10.1111/hsc.13349 · Cited by 6 · see citing works
Jülicher, P., O’Kane, M., Price, C. P., Christenson, R. H., & John, A. S. (2021) · Annals of Clinical Biochemistry International Journal of Laboratory Medicine · doi:10.1177/00045632211013852 · Cited by 5 · see citing works
Brown, V., Tran, H. N. Q., Williams, J. W., Laws, R., & Moodie, M. (2022) · BMC Public Health · doi:10.1186/s12889-022-13754-0 · Cited by 5 · see citing works
Pilz, M. J., Seyringer, S., Singer, S., Ioannidis, G., Sykiotis, G. P., Arrarás, J. I., Husson, O., Iakovou, I. P., Fanetti, G., Führer, D., et al. (2024) · Thyroid · doi:10.1089/thy.2024.0396 · Cited by 5 · see citing works
Bavor, C., Brotherton, J. M. L., Smith, M. A., Prang, K.-H., McDermott, T. L., Rankin, N. M., Zammit, C. M., Jennett, C. J., Sultana, F., Machalek, D. A., et al. (2023) · BMC Health Services Research · doi:10.1186/s12913-023-10040-6 · Cited by 4 · see citing works
The Costs of Implementing a Conversation Aid for Uterine Fibroids in Multiple Health Care Settings
Acquilano, S. C., Forcino, R. C., Schubbe, D. C., Engel, J., Tomaino, M., Johnson, L. C., Durand, M., & Elwyn, G. J. (2023) · Medical Care · doi:10.1097/mlr.0000000000001897 · Cited by 4 · see citing works
Mapping Organizational Theory With SCRIPTS
Trinidad, J. E. (2025) · Sociology Compass · doi:10.1111/soc4.70059 · Cited by 3 · see citing works
Carter, H. E., Donovan, T., White, N. M., Lee, X. J., Brown, C., Graves, N., McPhail, S., Cardona, M., White, B. P., Willmott, L., et al. (2025) · Age and Ageing · doi:10.1093/ageing/afaf280 · Cited by 3 · see citing works
Amritlal, S. T., Chandler, R., Ahari, A. M., Paterson, L., Avery, A. J., Ashcroft, D. M., Chuter, A., & Elliott, R. A. (2025) · PharmacoEconomics · doi:10.1007/s40273-025-01567-z · Cited by 3 · see citing works
Cost-effectiveness of an educational intervention to reduce potentially inappropriate medication
Rantsi, M., Pitkälä, K. H., Kautiainen, H., Hyttinen, V., & Kankaanpää, E. (2022) · Age and Ageing · doi:10.1093/ageing/afac112 · Cited by 3 · see citing works
Wynne, O., Szewczyk, Z., Hollis, J., Farragher, E., Doherty, E. L., Tully, B., Paolucci, F., Gillham, K. E., Reeves, P., Wiggers, J. H., et al. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00420-8 · Cited by 3 · see citing works
Early Rehabilitation Bundle in a Canadian PICU: Cost Analysis of Implementation in 2018–2020
Gertsman, S., Pavalagantharajah, S., Falk, L., Borhan, S., Kennedy, K., Thabane, L., Xie, F., Cupido, C. M., & Choong, K. L. M. (2025) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000003806 · Cited by 3 · see citing works
Rejon, C. S., Hollingworth, W., McLeod, H., Opmeer, B. C., & Luyt, K. (2023) · Research Square · doi:10.21203/rs.3.rs-2678140/v1 · Cited by 3 · see citing works
Rejon, C. S., Hollingworth, W., McLeod, H., Opmeer, B. C., & Luyt, K. (2023) · Research Square · doi:10.21203/rs.3.rs-2678140/v2 · Cited by 3 · see citing works
Zomer, E., Tan, E. J., Kim, J., Turner, T., Cadilhac, D. A.-M., Hill, K. M., Zoungas, S., McGloughlin, S., Le, L. L., & Millard, T. N. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5382385 · Cited by 3 · see citing works
Economic Considerations in Oral Health: Bridging Gaps, Broadening Horizons
Benzian, H., & Schwendicke, F. (2024) · Journal of Dental Research · doi:10.1177/00220345241286462 · Cited by 2 · see citing works
Blackburn, N. A., Philbrick, S. M., Patel, S. V., Cance, J. D., Elek, E., Gibbons, B., Khan, S. S., Lambdin, B. H., & Graham, P. (2025) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2025.102610 · Cited by 2 · see citing works
Resource use and in-hospital costs after aneurysmal subarachnoid hemorrhage in the Netherlands
Mulder, F., van Dijck, J. T. J. M., Vreeburg, R. J. G., Engels, C. C., & Moojen, W. A. (2025) · Brain and Spine · doi:10.1016/j.bas.2025.104400 · Cited by 2 · see citing works
Szewczyk, Z., MACDONALD, H. M., Pinheiro, M. B., Nettlefold, L., Sims‐Gould, J., & McKay, H. A. (2025) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-025-01826-4 · Cited by 2 · see citing works
Brundisini, F. K., Zomahoun, H. T. V., Légaré, F., Rhéault, N., Uwizeye, C. B., Massougbodji, J., Gogovor, A., Tchoubi, S., Assan, O. Q., & Laberge, M. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-050838 · Cited by 2 · see citing works
Keim‐Malpass, J., Heysell, S. K., Thomas, T. A., Lobo, J. M., Mpagama, S. G., Muzoora, C. K., & Moore, C. C. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20065041 · Cited by 2 · see citing works
Jabbar, S. A., Frödin, M., Wikström, E., Gillespie, B. M., Gyllensten, H., & Andersson, A. E. (2025) · Antimicrobial Resistance and Infection Control · doi:10.1186/s13756-025-01573-y · Cited by 2 · see citing works
Bartels, S. M., Phan, H. T. T., Hutton, H. E., Nhan, D. T., Sripaipan, T., Chen, J. S., Rossi, S. L., Ferguson, O., Nong, H. T. T., Nguyen, N. T. K., et al. (2024) · Implementation Science · doi:10.1186/s13012-024-01368-6 · Cited by 2 · see citing works
The Need to Conduct Studies on Economic Evaluation of Nursing Care in Iran
Nejatian, A. (2022) · Iran Journal of Nursing · doi:10.32598/ijn.35.139.3158 · Cited by 1 · see citing works
Michaud, T. L., Hill, J. L., Heelan, K. A., Fiechtner, L. G., Ruggiero, C. F., French, D. D., Smith, J. D., Salmon, J., Lindros, J., Jelalian, E., et al. (2025) · Childhood Obesity · doi:10.1177/21532176251389760 · Cited by 1 · see citing works
- Authors
- Roberts, S. L. E., Healey, A., & Sevdalis, N.
- Affiliations
- King's College London (GB)
- Corresponding author
- Roberts, Sarah Louise Elin
- Venue
- Implementation Science
- Published
- 2019-07-15
- Volume
- 14
- Issue
- 1
- Pages
- 72-72
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC6631608
- OpenAlex
- W2959993899
- Semantic Scholar
- 7ec32b728ccd1fb347c955b4d846cc3dc610e90b
- Cited by
- 161
- References
- 41
- Citations by year
- 2026: 14 · 2025: 20 · 2024: 22 · 2023: 26 · 2022: 30 · 2021: 26 · 2020: 21 · 2019: 1
- FWCI
- 5.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Implementation Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Portal (King's College London) (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- Funders
- National Institute for Health Research
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
- Keywords
- health economic evaluation, implementation research, improvement science, cost-effectiveness analysis, nurse-led care, implementation costs, cost-utility analysis, cost-consequence analysis, hospital care delivery
- MeSH terms
- Implementation Science, Cost-Benefit Analysis, Delivery of Health Care, Humans, Models, Economic, Evidence-Based Practice, Quality Improvement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
