Cited by 146Health Systems, Economic Evaluations, Quality of LifeEvaluation and Performance AssessmentHealth Policy Implementation Science

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.

Evaluating policy and service interventions: framework to guide selection and interpretation of study end points

Lilford, R. J., Chilton, P. J., Hemming, K., Girling, A. J., Taylor, C. A., Barach, P.·2010·BMJ

DOI 10.1136/bmj.c4413

Open this paper in CoChat — free account

Abstract

The effect of many cost effective policy and service interventions cannot be detected at the level of the patient. This new framework could help improve the design (especially choice of primary end point) and interpretation of evaluative studies

Integrity signals

Registry

Crossref · 10.1136/bmj.c4413

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995
Incidence of Adverse Events and Negligence in Hospitalized Patients
Brennan, T. A., Leape, L. L., Laird, N. M., Hebert, L. E., Localio, A. R., Lawthers, A. G., Newhouse, J. P., Weiler, P. C., & Hiatt, H. H. (1991) · New England Journal of Medicine · doi:10.1056/nejm199102073240604 · Cited by 4,631
An Intervention to Decrease Catheter-Related Bloodstream Infections in the ICU
Pronovost, P., Needham, D., Berenholtz, S., Sinopoli, D., Chu, H., Cosgrove, S., Sexton, B., Hyzy, R., Welsh, R., Roth, G., et al. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa061115 · Cited by 4,387
Effect of Reducing Interns' Work Hours on Serious Medical Errors in Intensive Care Units
Landrigan, C. P., Rothschild, J. M., Cronin, J. W., Kaushal, R., Burdick, E., Katz, J. T., Lilly, C. M., Stone, P. H., Lockley, S. W., Bates, D. W., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejmoa041406 · Cited by 1,760 · cited 1×
“However, as the signal (change in outcome due to intervention) diminishes in relation to the noise (changes in outcome due to uncontrolled sources of variation), a study based on process measurement will become more cost effective. Such was the case in Landrigan’s study of the effects of fatigue on the quality of care…”
Background
A look into the nature and causes of human errors in the intensive care unit
Donchin, Y., Gopher, D., Olin, M., Badihi, Y., Bieski, M., Sprung, C. L., Pizov, R., & Cotev, S. (1995) · Critical Care Medicine · doi:10.1097/00003246-199502000-00015 · Cited by 838
Effect of pneumonia case management on mortality in neonates, infants, and preschool children: a meta-analysis of community-based trials.
Sazawal, S., & Black, R. (2003) · Lancet. Infectious Diseases (Print) · doi:10.1016/s1473-3099(03)00737-0 · Cited by 485
Publication Bias: The Problem That Won't Go Away
DICKERSIN, K., & MIN, Y. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26343.x · Cited by 427
Determinants of the intracluster correlation coefficient in cluster randomized trials: the case of implementation research
Campbell, M. K., Fayers, P. M., & Grimshaw, J. M. (2005) · Clinical Trials · doi:10.1191/1740774505cn071oa · Cited by 275
Ethical issues in the design and conduct of cluster randomised controlled trials
Edwards, S., Braunholtz, D., Lilford, R. J., & Stevens, A. J. (1999) · BMJ · doi:10.1136/bmj.318.7195.1407 · Cited by 256
Self-citation
Methods to reduce the impact of intraclass correlation in group-randomized trials.
Murray, D. M., & Blitstein, J. L. (2003) · Evaluation Review · doi:10.1177/0193841x02239019 · Cited by 221

Flagged first, then by citations

Show all 31 references

Sorted by citations

Improving the health and welfare of people who live in slums
Lilford, R. J., Oyebode, O., Satterthwaite, D., Melendez-Torres, G. J., Chen, Y.-F., Mberu, B., Watson, S. I., Sartori, J., Ndugwa, R., Caiaffa, W., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(16)31848-7 · Cited by 305
Self-citation
Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation
Benning, A., Ghaleb, M. A., Suokas, A. K., Dixon‐Woods, M., Dawson, J., Barber, N., Franklin, B. D., Girling, A. J., Hemming, K., Carmalt, M., et al. (2011) · BMJ · doi:10.1136/bmj.d195 · Cited by 233
Self-citation
Evaluating the impact of prediction models: lessons learned, challenges, and recommendations
Kappen, T. H., van Klei, W. A., van Wolfswinkel, L., Kalkman, C. J., Vergouwe, Y., & Moons, K. G. M. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0033-6 · Cited by 215
“Accordingly, preceding decision analytic studies help in separating the chaff from the wheat [45].”
Background
Challenges, solutions and future directions in the evaluation of service innovations in health care and public health
Raine, R., Fitzpatrick, R. M., Barratt, H., Bevan, G., Black, N. A., Boaden, R., Bower, P., Campbell, M., Denis, J., Devers, K. J., et al. (2016) · Health Services and Delivery Research · doi:10.3310/hsdr04160 · Cited by 197
Self-citation
Changes in adverse event rates in hospitals over time: a longitudinal retrospective patient record review study
Baines, R. J., Langelaan, M., de Bruijne, M. C., Asscheman, H., Spreeuwenberg, P. M. M., van de Steeg, L., Siemerink, K. M., van Rosse, F., Broekens, M. A., & Wagner, C. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001126 · Cited by 177
Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase
Benning, A., Dixon‐Woods, M., Nwulu, U., Ghaleb, M. A., Dawson, J., Barber, N., Franklin, B. D., Girling, A. J., Hemming, K., Carmalt, M., et al. (2011) · BMJ · doi:10.1136/bmj.d199 · Cited by 166
Self-citation
Improving clinical handover between intensive care unit and general ward professionals at intensive care unit discharge
van Sluisveld, N., Hesselink, G., van der Hoeven, J. G., Westert, G., Wollersheim, H., & Zegers, M. (2015) · Intensive Care Medicine · doi:10.1007/s00134-015-3666-8 · Cited by 130
“These reasons reflect the difficulty of demonstrating the effectiveness of complex quality improvement interventions, as has been mentioned in several publications [60–62].”
Result
Recommendations for Evaluation of Health Care Improvement Initiatives
Parry, G. J., Carson-Stevens, A., Luff, D. F., McPherson, M. E., & Goldmann, D. A. (2013) · Academic Pediatrics · doi:10.1016/j.acap.2013.04.007 · Cited by 124
Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research: A meta-epidemiological study
Ayorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y. (2020) · PLoS ONE · doi:10.1371/journal.pone.0227580 · Cited by 121
Self-citation
A prospective study of paediatric cardiac surgical microsystems: assessing the relationships between non-routine events, teamwork and patient outcomes
Schraagen, J. M., Schouten, T. J., Smit, M., Haas, F., van der Beek, D., van de Ven, J., & Barach, P. R. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs.2010.048983 · Cited by 104
Self-citation
Secular trends and evaluation of complex interventions: the rising tide phenomenon
Chen, Y., Hemming, K., Stevens, A. J., & Lilford, R. J. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004372 · Cited by 102
Self-citation
Public Health's Next Step in Advancing Equity: Re-evaluating Epistemological Assumptions to Move Social Determinants From Theory to Practice
Golden, T. L., & Wendel, M. L. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00131 · Cited by 110
Investigating the use of data-driven artificial intelligence in computerised decision support systems for health and social care: A systematic review
Cresswell, K., Callaghan, M., Khan, S., Sheikh, Z., Mozaffar, H., & Sheikh, A. (2020) · Health Informatics Journal · doi:10.1177/1460458219900452 · Cited by 87
“21 These may include qualitative evaluations, either as stand-alone studies or embedded with RCTs to investigate mechanisms of action.”
Background
Understanding Contrasting Approaches to Nationwide Implementations of Electronic Health Record Systems: England, the USA and Australia
Morrison, Z., Robertson, A., Cresswell, K., Crowe, S., & Sheikh, A. (2011) · Journal of Healthcare Engineering · doi:10.1260/2040-2295.2.1.25 · Cited by 81
“There is also a clear need for work that focuses on building an evidence base for the benefits of national EHR implementations [58, 59].”
Background
A systematic review of instruments that assess the implementation of hospital quality management systems
Groene, O., Botje, D., Suñol, R., Gómez, M. A. L., & Wagner, C. (2013) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzt058 · Cited by 76
The FEeding Support Team (FEST) randomised, controlled feasibility trial of proactive and reactive telephone support for breastfeeding women living in disadvantaged areas
Hoddinott, P., Craig, L., Maclennan, G., Boyers, D., & Vale, L. (2012) · BMJ Open · doi:10.1136/bmjopen-2011-000652 · Cited by 66
High Reliability Organizations and Surgical Microsystems: Re-engineering Surgical Care
Sánchez, J. A., & Barach, P. R. (2012) · Surgical Clinics of North America · doi:10.1016/j.suc.2011.12.005 · Cited by 64
Self-citation
Why and how to use mixed methods in primary health care research
Vedel, I., Kaur, N., Hong, Q. N., Sherif, R. E., Khanassov, V., Godard‐Sebillotte, C., Sourial, N., Yang, X. Q., & Pluye, P. (2018) · Family Practice · doi:10.1093/fampra/cmy127 · Cited by 58
Higher Stroke Unit Volume Associated With Improved Quality of Early Stroke Care and Reduced Length of Stay
Svendsen, M. L., Ehlers, L. H., Ingeman, A., & Johnsen, S. P. (2012) · Stroke · doi:10.1161/strokeaha.111.645184 · Cited by 54
Investigating the physical activity, health, wellbeing, social and environmental effects of a new urban greenway: a natural experiment (the PARC study)
Hunter, R. F., Adlakha, D., Cardwell, C. R., Cupples, M., Donnelly, M., Ellis, G., Gough, A., Hutchinson, G., Kearney, T., Longo, A., et al. (2021) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-021-01213-9 · Cited by 51
“These results pose several scientific and real-world implementation challenges that are too infrequently exposed in public heath intervention trials [60], including how to balance positive and negative results when primary and secondary outcomes are discordant [61].”
Background
Building a house on shifting sand: methodological considerations when evaluating the implementation and adoption of national electronic health record systems
Takian, A., Petrakaki, D., Cornford, T., Sheikh, A., & Barber, N. (2012) · BMC Health Services Research · doi:10.1186/1472-6963-12-105 · Cited by 51
“Indeed, choice of appropriate methods for such evaluations remains a contested issue [29,30].”
Background
The European HANDOVER Project: a multi-nation program to improve transitions at the primary care—inpatient interface
Philibert, I., & Barach, P. R. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001598 · Cited by 51
“interventions that are aimed at the population of organisations in the field.(25) 26 They emphasise that efforts”
BackgroundSelf-citation
Conducting a multicentre and multinational qualitative study on patient transitions
Johnson, J. K., Barach, P. R., & Vernooij-Dassen, M. J. F. J. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001197 · Cited by 49
“We also used a series of quantitative methods to assess the cost effectiveness of handover training.(4) From the initial funding of the HANDOVER Project, the Project’s leadership agreed that, in addition to the research activities to meet the investigational aims of the project, there would be a separate set of activi…”
MethodSelf-citation
Cost-effectiveness of health care service delivery interventions in low and middle income countries: a systematic review
Watson, S., Sahota, H., Taylor, C. A., Chen, Y., & Lilford, R. J. (2018) · Global Health Research and Policy · doi:10.1186/s41256-018-0073-z · Cited by 49
“A clinical intervention is applied directly to a patient whereas a service delivery intervention is applied to the organisation of a clinical intervention and its delivery [5].”
BackgroundInfluentialSelf-citation
HEADROOM APPROACH TO DEVICE DEVELOPMENT: CURRENT AND FUTURE DIRECTIONS
Girling, A. J., Lilford, R. J., Cole, A., & Young, T. P. (2015) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462315000501 · Cited by 48
Self-citation
Systems of service: reflections on the moral foundations of improvement
Davidoff, F. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs.2010.046177 · Cited by 46
“Crucial differences among the kind of measurements needed for improvement, accountability and research purposes have been well described,(32) as has the importance of using the appropriate primary outcome measurement in evaluating policy and service interventions.(12) Sophisticated frameworks for understanding context…”
Background
Assessing health system interventions: key points when considerting the value of randomizatioin
English, M., Schellenberg, J., & Todd, J. (2011) · Bulletin of the World Health Organization · doi:10.2471/blt.11.089524 · Cited by 45
Lay support for pregnant women with social risk: a randomised controlled trial
Kenyon, S. L., Jolly, K. B., Hemming, K., Hope, L., Blissett, J. M., Dann, S.-A., Lilford, R. J., & MacArthur, C. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-009203 · Cited by 46
“Identified benefit problem 154 (23) 160 (24)”
Self-citation
Publication and related biases in health services research: a systematic review of empirical evidence
Ayorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01010-1 · Cited by 45
“ables and outcome variables) along a long service delivery causal chain [9].”
BackgroundSelf-citation
Randomised health services studies
Hakama, M., Malila, N. K., & Dillner, J. (2012) · International Journal of Cancer · doi:10.1002/ijc.27561 · Cited by 43
Stakeholder perspectives on handovers between hospital staff and general practitioners: an evaluation through the microsystems lens
Göbel, B., Zwart, D. L. M., Hesselink, G., Pijnenborg, L., Barach, P. R., Kalkman, C. J., & Johnson, J. K. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001192 · Cited by 44
Self-citation
Implementation of a ‘care bundle’ improves the management of patients admitted to hospital with decompensated cirrhosis
Dyson, J. K., Rajasekhar, P. T., Wetten, A., Hamad, A. H., Ng, S. C., Paremal, S., Baqai, M. F., Lamb, C. A., Masson, S., Hudson, M., et al. (2016) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.13806 · Cited by 41
“This work describes the process of successfully introducing evidencebased care that reduces variability and it is logical to infer that if care is delivered at a consistently high level then a morbidity and mortality benefit will be observed if larger numbers were included.(15) Larger prospective controlled studies ma…”
Background
Effectiveness and cost-effectiveness of a loyalty scheme for physical activity behaviour change maintenance: results from a cluster randomised controlled trial
on behalf of the Physical Activity Loyalty (PAL) Study team, Hunter, R. F., Murray, J., Gough, A., Tang, J., Patterson, C., French, D., McIntosh, E., Xin, Y., & Kee, F. (2018) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-018-0758-1 · Cited by 40
Evaluation of a predevelopment service delivery intervention: an application to improve clinical handovers
Yao, G., Novielli, N., Manaseki‐Holland, S., Chen, Y., van der Klink, M. R., Barach, P. R., Chilton, P. J., & Lilford, R. J. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001210 · Cited by 39
Self-citation
The bird’s-eye perspective: how do district health managers experience the impact of family physicians within the South African district health system? A qualitative study
Pressentin, K. B. V., Mash, R. J., Baldwin-Ragaven, L., Botha, R., Govender, I., & Steinberg, W. J. (2017) · South African Family Practice · doi:10.1080/20786190.2017.1348047 · Cited by 38
Measuring the Performance of Public Services: Principles and Practice
Pidd, M. (2012) · Cited by 38
Comparing the use of direct observation, standardized patients and exit interviews in low- and middle-income countries: a systematic review of methods of assessing quality of primary care
Aujla, N., Chen, Y., Samarakoon, Y. M., Wilson, A. C., Grolmusova, N., Ayorinde, A., Hofer, T. P., Griffiths, F. E., Brown, C., Gill, P. S., et al. (2020) · Health Policy and Planning · doi:10.1093/heapol/czaa152 · Cited by 36
“In this paper, we concentrate on process, which can be broken down into processes carried out at the system level, such as use of audit and feedback or improving staff morale, and clinical processes impacting directly on patients, such as questions asked to make a diagnosis or prescribe a treatment (Lilford et al., 20…”
BackgroundSelf-citation
The Link Between Participation in Management Decision-Making and Quality of Patient Care as Perceived by Chinese Doctors
Liu, C., Bartram, T., Casimir, G., & Leggat, S. G. (2014) · Public Management Review · doi:10.1080/14719037.2014.930507 · Cited by 34
“Continued research in this area is important as it will inform and influence Chinese health policy and the effective management of Chinese hospitals (Lilford et al. 2010).”
Background
DUQuE quality management measures: associations between quality management at hospital and pathway levels
Wagner, C., Groene, O., Thompson, C., DerSarkissian, M., Klazinga, N., Arah, O., & Suñol, R. (2014) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzu020 · Cited by 33
“activities of professionals at the sharp end of care processes [4].”
Background
The Primary-Secondary Care Partnership to Improve Outcomes in Chronic Kidney Disease (PSP-CKD) Study: A Cluster Randomized Trial in Primary Care
Major, R., Brown, C., Shepherd, D. J., Rogers, S., Pickering, W. P., Warwick, G. L., Barber, S. R., Ashra, N., Morris, T., & Brunskill, N. J. (2019) · Journal of the American Society of Nephrology · doi:10.1681/asn.2018101042 · Cited by 34
Impact of electronic health records on predefined safety outcomes in patients admitted to hospital: a scoping review
Subbe, C. P., Tellier, G., & Barach, P. R. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047446 · Cited by 36
Self-citation
Process evaluation for the FEeding Support Team (FEST) randomised controlled feasibility trial of proactive and reactive telephone support for breastfeeding women living in disadvantaged areas
Hoddinott, P., Craig, L., MacLennan, G., Boyers, D., & Vale, L. (2012) · BMJ Open · doi:10.1136/bmjopen-2012-001039 · Cited by 32
Economic analysis of service and delivery interventions in health care
Sutton, M., Garfield-Birkbeck, S., Martin, G. P., Meacock, R., Stephen Morris, A., Sculpher, M. J., Street, A., Watson, S., & Lilford, R. J. (2018) · Health Services and Delivery Research · doi:10.3310/hsdr06050 · Cited by 31
Self-citation
Traspaso de información en Medicina Intensiva
Sirgo, G., Fernández, M. C., Vidal, F. G., Arias, M. G., Peña, M. S. H., Ayerdi, B. A., Andrés, E. B., Sellés, A. F., García, P., García, M. G., et al. (2018) · Medicina Intensiva · doi:10.1016/j.medin.2017.12.002 · Cited by 31
The Influence of Family Physicians Within the South African District Health System: A Cross-Sectional Study
Pressentin, K. B. V., Mash, R. J., Baldwin-Ragaven, L., Botha, R. P. G., Govender, I., Steinberg, W. J., & Esterhuizen, T. M. (2018) · The Annals of Family Medicine · doi:10.1370/afm.2133 · Cited by 31
Is the maturity of hospitals' quality improvement systems associated with measures of quality and patient safety?
Groene, O., Mora, N., Thompson, A. G. H., Saez, M., Casas, M., & Suñol, R. (2011) · BMC Health Services Research · doi:10.1186/1472-6963-11-344 · Cited by 28
“Attribution of quality improvement activity to outcomes might be supported by further breaking down assessment of processes into generic management processes, targeted management processes and clinical processes [41].”
Background
Methods for the economic evaluation of changes to the organisation and delivery of health services: principal challenges and recommendations
Meacock, R. (2018) · Health Economics Policy and Law · doi:10.1017/s1744133118000063 · Cited by 27
“The causal chain between intervention and effect may be more diluted (Lilford et al., 2010), and the effectiveness of a service intervention will be heavily dependent on the interplay of other areas of the system.”
Background
Challenges in Assessing the Process–Outcome Link in Practice
Parast, L. M., Doyle, B. J., Damberg, C. L., Shetty, K. D., Ganz, D. A., Wenger, N. S., & Shekelle, P. (2015) · Journal of General Internal Medicine · doi:10.1007/s11606-014-3150-0 · Cited by 26
Economic impact of electronic prescribing in the hospital setting: A systematic review
Ahmed, Z., Barber, N., Jani, Y. H., Garfield, S., & Franklin, B. D. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2015.11.008 · Cited by 24
Improving Primary Care Medication Processes by Using Shared Electronic Medication Plans in Switzerland: Lessons Learned From a Participatory Action Research Study
Bugnon, B., Geissbühler, A., Bischoff, T., Bonnabry, P., & von Plessen, C. (2020) · JMIR Formative Research · doi:10.2196/22319 · Cited by 22
“We based our model on Lilford et al’s [61] approach to mapping policy and service interventions with regard to structures, processes, outcomes, and intervening variables .”
Handover in Intensive Care
Sirgo, G., Fernández, M. C., Vidal, F. G., Arias, M. G., Peña, M. S. H., Ayerdi, B. A., Andrés, E. B., Sellés, A. F., García, P., García, M. G., et al. (2018) · Medicina Intensiva (English Edition) · doi:10.1016/j.medine.2017.12.008 · Cited by 21
Secondary analysis and literature review of community rehabilitation and intermediate care: an information resource
Ariss, S., Enderby, P., Smith, T., Nancarrow, S., Bradburn, M. J., Harrop, D., Parker, S. G., McDonnell, A., Dixon, S., Ryan, T., et al. (2015) · Health Services and Delivery Research · doi:10.3310/hsdr03010 · Cited by 21
Patient safety and rapid response systems
Hillman, K., Lilford, R. J., & Braithwaite, J. (2014) · The Medical Journal of Australia · doi:10.5694/mja14.01260 · Cited by 20
Self-citation
Self-care programmes for people living with leprosy: a scoping review
Ilozumba, O., & Lilford, R. J. (2021) · Leprosy Review · doi:10.47276/lr.92.4.317 · Cited by 19
Self-citation
The Long and Winding Road: An Independent Evaluation of the Implementation and Adoption of the National Health Service Care Records Service (NHS CRS) in Secondary Care in England
Cresswell, K. M., Ali, M., Avery, A. J., Barber, N., Cornford, T., Crowe, S., Fernando, B., Jacklin, A., Jani, Y. H., Klecuń, E., et al. (2011) · Figshare · Cited by 18
Evaluations of water, sanitation and hygiene interventions should not use diarrhoea as (primary) outcome
Watson, S., Rego, R. T. T., Hofer, T. P., & Lilford, R. J. (2022) · BMJ Global Health · doi:10.1136/bmjgh-2022-008521 · Cited by 19
Self-citation
Measuring and improving comprehensive pediatric cardiac care: Learning from continuous quality improvement methods and tools
Barach, P. R., & Kleinman, L. C. (2018) · Progress in Pediatric Cardiology · doi:10.1016/j.ppedcard.2018.02.010 · Cited by 16
Self-citation
Investigating the impact of a 20 miles per hour speed limit intervention on road traffic collisions, casualties, speed and volume in Belfast, UK: 3 year follow-up outcomes of a natural experiment
Hunter, R. F., Cleland, C., Busby, J., Nightingale, G., Kee, F., Williams, A. J., Kelly, P., Kelly, M. P., Milton, K. E., Kokka, K., et al. (2022) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2022-219729 · Cited by 15
Evidence‐based inpatient handovers: a literature review and research agenda
Scott, P., Ross, P., & Prytherch, D. (2012) · Clinical Governance An International Journal · doi:10.1108/14777271211200710 · Cited by 15
Policy and service delivery proposals to improve primary care services in low-income and middle-income country cities
Lilford, R. J., Daniels, B., McPake, B., Bhutta, Z. A., Mash, R. J., Griffiths, F. E., Omigbodun, A. O., Pinto, E. P., Jain, R., Asiki, G., et al. (2025) · The Lancet Global Health · doi:10.1016/s2214-109x(24)00536-9 · Cited by 18
Self-citation
Association between Exercise and Metabolic Syndrome in Koreans
Lee, S. H., Lee, Y., Seo, J. H., & Kim, Y. A. (2018) · Journal of Obesity & Metabolic Syndrome · doi:10.7570/jomes.2018.27.2.117 · Cited by 14
Addressing barriers for change in clinical practice
Barach, P. R. (2016) · Cambridge University Press eBooks · doi:10.1017/cbo9781316218563.019 · Cited by 13
Self-citation
Quality and safety in the transitional care of the elderly (phase 2): the study protocol of a quasi-experimental intervention study for a cross-level educational programme
Storm, M., Groene, O., Testad, I., Dyrstad, D. N., Heskestad, R. N., & Aase, K. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005962 · Cited by 13
Protocol for evaluation of the cost-effectiveness of ePrescribing systems and candidate prototype for other related health information technologies
Lilford, R. J., Girling, A. J., Sheikh, A., Coleman, J. J., Chilton, P. J., Burn, S. L., Jenkinson, D., Blake, L., & Hemming, K. (2014) · BMC Health Services Research · doi:10.1186/1472-6963-14-314 · Cited by 13
“Conclusion We present a method to deal with the “inconvenient truth” [5] that occurs when complex generic service delivery interventions must be assessed for cost-effectiveness.”
Self-citation
Towards a learning system for pediatric outcomes: Harvesting meaning from evidence
Kleinman, L. C., & Barach, P. R. (2018) · Progress in Pediatric Cardiology · doi:10.1016/j.ppedcard.2018.05.002 · Cited by 12
Self-citation
Supply-side and demand-side factors affecting allopathic primary care service delivery in low-income and middle-income country cities
Lilford, R. J., Daniels, B., McPake, B., Bhutta, Z. A., Mash, R. J., Griffiths, F. E., Omigbodun, A. O., Pinto, E. P., Jain, R., Asiki, G., et al. (2025) · The Lancet Global Health · doi:10.1016/s2214-109x(24)00535-7 · Cited by 11
Self-citation
What can artefact analysis tell us about patient transitions between the hospital and primary care? Lessons from the HANDOVER project
Johnson, J. K., Arora, V. M., Barach, P. R., & Collaborative, O. B. O. T. E. H. R. (2013) · European Journal of General Practice · doi:10.3109/13814788.2013.819850 · Cited by 11
“We also used a series of quantitative methods to assess the cost eff ectiveness of handover training (11,12).”
MethodSelf-citation
Rethinking COVID-19 in children: Lessons learned from pediatric viral and inflammatory cardiovascular diseases
Barach, P. R., & Lipshultz, S. E. (2020) · Progress in Pediatric Cardiology · doi:10.1016/j.ppedcard.2020.101233 · Cited by 11
Self-citation
Increasing specialist intensity at weekends to improve outcomes for patients undergoing emergency hospital admission: the HiSLAC two-phase mixed-methods study
Bion, J., Aldridge, C., Beet, C., Boyal, A., Chen, Y., Clancy, M. J., Girling, A. J., Hofer, T. P., Lord, J. L., Mannion, R., et al. (2021) · Health Services and Delivery Research · doi:10.3310/hsdr09130 · Cited by 12
Self-citation
Two decades of research on innovative models of care delivery for patients with heart failure: the end or just the beginning?
Rahimi, K., Patel, A. A., & MacMahon, S. W. (2012) · PubMed · Cited by 10
Qualitative analysis of multi-disciplinary round-table discussions on the acceleration of benefits and data analytics through hospital electronic prescribing (ePrescribing) systems
Cresswell, K. M., Coleman, J. J., Smith, P. A., Swainson, C. P., Slee, A., & Sheikh, A. (2016) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v23i2.178 · Cited by 12
Readmitting Children with Heart Failure: the Importance of Communication, Coordination, and Continuity of Care
Barach, P. R., & Lipshultz, S. E. (2016) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2016.07.027 · Cited by 10
Self-citation
Retrospective evaluation of an intervention based on training sessions to increase the use of control charts in hospitals
Kudrna, L., Bird, P., Hemming, K., Quinn, L., Schmidtke, K. A., & Lilford, R. J. (2022) · BMJ Quality & Safety · doi:10.1136/bmjqs-2021-013514 · Cited by 10
Self-citation
Does Quality Improvement Face a Legitimacy Crisis? Poor Quality Studies, Small Effects
Groene, O. (2011) · Journal of Health Services Research & Policy · doi:10.1258/jhsrp.2011.011022 · Cited by 10
Toward a Learning System for ERAS: Embedding Implementation and Learning Evaluation
Ramaswamy, R., & Barach, P. R. (2020) · Springer eBooks · doi:10.1007/978-3-030-33443-7_39 · Cited by 9
Self-citation
Electronic prescribing systems in hospitals to improve medication safety: a multimethods research programme
Sheikh, A., Coleman, J. J., Chuter, A., Williams, R., Lilford, R. J., Slee, A., Morrison, Z., Cresswell, K. M., Robertson, A. R. R., Slight, S. P., et al. (2022) · Programme Grants for Applied Research · doi:10.3310/ksrs2009 · Cited by 12
Self-citation
When is early septal myectomy in children with hypertrophic cardiomyopathy justified?
Bansal, N., Barach, P. R., Amdani, S. M., & Lipshultz, S. E. (2018) · Translational Pediatrics · doi:10.21037/tp.2018.09.08 · Cited by 9
Self-citation
Content counts, but context makes the difference in developing expertise: a qualitative study of how residents learn end of shift handoffs
Rattray, N. A., Ebright, P. R., Flanagan, M., Militello, L. G., Barach, P. R., Franks, Z., Rehman, S. U., Gordon, H. S., & Frankel, R. M. (2018) · BMC Medical Education · doi:10.1186/s12909-018-1350-8 · Cited by 11
Self-citation
Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial
Jenniskens, K., Lagerweij, G. R., Naaktgeboren, C. A., Hooft, L., Moons, K. G. M., Poldervaart, J. M., Koffijberg, H., & Reitsma, J. B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.07.010 · Cited by 9
Barriers and facilitators of family doctor contract services in caring for disabled older adults in Beijing, China: a mixed methods study
Zhang, Z., Zhang, R., Peng, Y., Zhai, S., Zhang, J., Jin, Q., Zhou, J., Li, H., & Chen, J. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-070130 · Cited by 8
Is quality affordable for community health systems? Costs of integrating quality improvement into close-to-community health programmes in five low-income and middle-income countries
Kumar, M. B., Madan, J. J., Mireku, M., Limato, R., Ndima, S. D., Kea, A. Z., Chikaphupha, K. R., Barasa, E., & Taegtmeyer, M. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2019-001390 · Cited by 8
“Measuring and attributing the downstream benefits of a service delivery intervention that are intrinsically valuable to a decision maker or population is challenging.(20) 68–72 Adding to the challenges of potential confounding, in ‘Step Six’ of the intervention (figure 1), QI teams have the freedom to design and test…”
Background
How far does family physician supply correlate with district health system performance?
Dyers, R., Mash, R. J., & Naledi, T. (2015) · African Journal of Primary Health Care & Family Medicine · doi:10.4102/phcfm.v7i1.796 · Cited by 8
A multifaceted intervention to improve diagnosis and early management of hospitalised patients with suspected acute brain infections in Brazil, India, and Malawi: an international multicentre intervention study
Singh, B., Lipunga, G., Thangavelu, P., Dhar, S., Cronemberger, L. F., Abhilash, K. P. P., Abraham, A. M., de Brito, C. A. A., Ferreira, M. L. B., Chandrashekar, N., et al. (2025) · The Lancet · doi:10.1016/s0140-6736(25)00263-6 · Cited by 8
“For the primary outcomes the analysis included a comparison of percentage of patients diagnosed before versus after the intervention, as well as interrupted time series analysis.”
Self-citation
Designing high-reliability healthcare teams
Barach, P. R. (2016) · International Conference on Information and Digital Technologies (IDT) · doi:10.1109/dt.2016.7557144 · Cited by 7
Self-citation
The impact of family physician supply on district health system performance, clinical processes and clinical outcomes in the Western Cape Province, South Africa (2011–2014)
Chinhoyi, R. L., Zunza, M., & Pressentin, K. B. V. (2018) · African Journal of Primary Health Care & Family Medicine · doi:10.4102/phcfm.v10i1.1442 · Cited by 7
Implementation Science: Translating Research into Practice for Sustained Impact
Aarons, G. A., Sklar, M., & Sevdalis, N. (2017) · Springer eBooks · doi:10.1007/978-3-319-44010-1_18 · Cited by 6
An opportunistic evaluation of a routine service improvement project to reduce falls in hospital
Sheppard, D., Clarke, E., Hemming, K., Martín, J., & Lilford, R. J. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06073-4 · Cited by 6
Self-citation
Estimating the Effect of Health Service Delivery Interventions on Patient Length of Stay: A Bayesian Survival Analysis Approach
Watson, S., Lilford, R. J., Sun, J., & Bion, J. (2021) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/rssc.12501 · Cited by 5
Self-citation
The bird’s-eye perspective: how do district health managers experience the impact of family physicians within the South African district health system? A qualitative study
Pressentin, K. B. V., Mash, R. J., Baldwin-Ragaven, L., Botha, R. P. G., Govender, I., & Steinberg, W. J. (2018) · South African Family Practice · doi:10.4102/safp.v60i1.4844 · Cited by 6
Failure to Rescue and Failure to Perceive Patients in Crisis
Subbe, C. P., & Barach, P. R. (2017) · Springer eBooks · doi:10.1007/978-3-319-44010-1_37 · Cited by 6
Self-citation
Enterprise Risk Management in Healthcare
Levett, J. M., Fasone, J. M., Smith, A. L., Labovitz, S. S., Labovitz, J., Mellott, S., & Dotan, D. B. (2017) · Springer eBooks · doi:10.1007/978-3-319-44010-1_6 · Cited by 5
Discussion of “Evidence-based Health Informatics: How Do We Know What We Know?”
Al-Shorbaji, N., Hanmer, L. A., Hussein, R., Magrabi, F., Moen, A., Moura, L. A., Park, H., & Scott, P. (2015) · Methods of Information in Medicine · doi:10.3414/me14-02-0119 · Cited by 5
Operationalising perioperative safety: learning to walk the talk
Chejfec‐Ciociano, J. M., Barach, P. R., & Gelb, A. W. (2025) · British Journal of Anaesthesia · doi:10.1016/j.bja.2025.06.018 · Cited by 4
Self-citation
Cluster randomised evaluation of a training intervention to increase the use of statistical process control charts for hospitals in England: making data count
Schmidtke, K. A., Kudrna, L., Quinn, L., Bird, P., Hemming, K., Venable, Z., & Lilford, R. J. (2024) · BMJ Quality & Safety · doi:10.1136/bmjqs-2024-017094 · Cited by 4
Self-citation
Publication and related bias in quantitative health services and delivery research: a multimethod study
Ayorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y. (2020) · Health Services and Delivery Research · doi:10.3310/hsdr08330 · Cited by 4
Self-citation
Development of a video-observation method for examining doctors’ clinical and interpersonal skills in a hospital outpatient clinic in Ibadan, Oyo State, Nigeria
Aujla, N., Ilori, T., Irabor, A. E., Obimakinde, A. M., Owoaje, E., Fayehun, O. A., Ajisola, M. M., Bolaji, S. O., Watson, S., Hofer, T. P., et al. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06491-4 · Cited by 4
“This paper focuses on the ‘process’ component of quality of care [10], which describes the clinical processes involving patients, such as the action of ordering a test or conducting an examination when necessary [11].”
BackgroundSelf-citation
Tools and Strategies for Continuous Quality Improvement and Patient Safety
Johnson, J. K., & Barach, P. R. (2017) · Springer eBooks · doi:10.1007/978-3-319-44010-1_9 · Cited by 4
Self-citation
Protocol for the health economic evaluation of increasing the weekend specialist to patient ratio in hospitals in England
Watson, S., Chen, Y., Bion, J., Aldridge, C., Girling, A. J., & Lilford, R. J. (2018) · BMJ Open · doi:10.1136/bmjopen-2016-015561 · Cited by 4
Self-citation
Key Issues in the Development of an Evidence-Based Stratified Surgical Patient Safety Improvement Information System: Experience From a Multicenter Surgical Safety Program
Yu, X., Han, W., Jiang, J., Wang, Y., Xin, S., Wu, S., Sun, H. J., Wang, Z., & Zhao, Y. (2019) · Journal of Medical Internet Research · doi:10.2196/13576 · Cited by 4
“These indicators, unlike process or surrogate safety indicators (eg, error or culture), are identifiable and quantifiable and are more appropriate as natural endpoints in the story of patient safety [22].”
Does Lowering Heart Rate Improve Outcomes in Children With Dilated Cardiomyopathy and Chronic Heart Failure?
Lipshultz, S. E., Barach, P. R., & Wilkinson, J. D. (2017) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2017.07.749 · Cited by 4
“Methods for selecting and interpreting study endpoints in the evaluation of policy and service interventions remain contested (8).”
BackgroundSelf-citation

Authors: Lilford, R. J., Chilton, P. J., Hemming, K., Girling, A. J., Taylor, C. A., & Barach, P.
Affiliations: University of Birmingham (GB)
University Medical Center Utrecht (NL)
Corresponding author: Lilford, R. J.
ORCID: Lilford, R. J.
Chilton, P. J.
Hemming, K.
Girling, A. J.
Barach, P.
Venue: BMJ
Publisher: BMJ
Published: 2010-08-27
Volume: 341
Issue: aug27 1
Pages: c4413-c4413
ISSN: 0007-1447 (ISSN-L)
0959-8138
1468-5833
DOI: 10.1136/bmj.c4413
OpenAlex: W1996547462
Cited by: 146
References: 31
Citations by year: 2026: 1 · 2025: 5 · 2024: 5 · 2023: 5 · 2022: 6 · 2021: 17 · 2020: 11 · 2019: 4 · 2018: 19 · 2017: 19 · 2016: 10 · 2015: 10 · 2014: 8 · 2013: 4 · 2012: 13
FWCI: 19.39
Type: journal article
Language: en
Open access: closed
Locations: BMJ (publishedVersion)
PubMed (publishedVersion)
University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
Funders: Engineering and Physical Sciences Research Council (GR/S29874/01, EP/F063822/1)
Topics: Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
Social Sciences › Public Administration › Evaluation and Performance Assessment
Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
Keywords: health policy evaluation, health service interventions, evaluation framework, cost-effectiveness
MeSH terms: Bayes Theorem, Cost-Benefit Analysis, Health Policy, Program Evaluation, Biomarkers, Organizational Policy, Outcome Assessment, Health Care, Process Assessment, Health Care, Publication Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Evaluating policy and service interventions: framework to guide selection and interpretation of study end points

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks