Research designs for studies evaluating the effectiveness of change and improvement strategies
Open accessAbstract
The methods of evaluating change and improvement strategies are not well described. The design and conduct of a range of experimental and non-experimental quantitative designs are considered. Such study designs should usually be used in a context where they build on appropriate theoretical, qualitative and modelling work, particularly in the development of appropriate interventions. A range of experimental designs are discussed including single and multiple arm randomised controlled trials and the use of more complex factorial and block designs. The impact of randomisation at both group and individual levels and three non-experimental designs (uncontrolled before and after, controlled before and after, and time series analysis) are also considered. The design chosen will reflect both the needs (and resources) in any particular circumstances and also the purpose of the evaluation. The general principle underlying the choice of evaluative design is, however, simple-those conducting such evaluations should use the most robust design possible to minimise bias and maximise generalisability.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The interventions may have been poorly developed, lacking appropriate theoretical basis and subsequent components to impact on the measured outcome domains.
“The interventions may have been poorly developed, lacking appropriate theoretical basis and subsequent components to impact on the measured outcome domains [82,83].”
From Interventions targeting social isolation in older people: a systematic review (2011) · cites this paper as [82]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1963) · Medical Entomology and Zoology · Cited by 15,085 · see citing works
(1979) · Medical Entomology and Zoology · Cited by 11,252 · see citing works
Quasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,107 · see citing works
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · see citing works
Clinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,951 · see citing works
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · see citing works
Effectiveness and Efficiency: Random Reflections on Health Services
L'Etang, H. (1973) · Public Health · doi:10.1016/s0033-3506(73)80082-4 · Cited by 1,925 · see citing works
Explanatory and Pragmatic Attitudes in Therapeutical Trials
Schwartz, D., & Lellouch, J. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.01.012 · Cited by 1,253 · see citing works
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236 · see citing works
Time-Series Analysis: Regression Techniques.
Pampel, F. C., & Ostrom, C. W. (1980) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2065616 · Cited by 499 · see citing works
Design and Analysis of Cluster Randomization Trials in Health Research.
Flynn, T. N. (2001) · International Journal of Epidemiology · doi:10.1093/ije/30.2.407-a · Cited by 482 · see citing works
Eccles, M. (2002) · BMJ · doi:10.1136/bmj.325.7370.941 · Cited by 354 · see citing works
Self-citationAnalysis of cluster randomized trials in primary care: a practical approach
Campbell, M. K., Mollison, J., Steen, N., Grimshaw, J. M., & Eccles, M. (2000) · Family Practice · doi:10.1093/fampra/17.2.192 · Cited by 327 · see citing works
Self-citationThe Estimation of Intraclass Correlation in the Analysis of Family Data
Donner, A. P., & Koval, J. J. (1980) · Biometrics · doi:10.2307/2530491 · Cited by 305 · see citing works
Eccles, M., Steen, N., Grimshaw, J., Thomas, L., McNamee, P., Soutter, J., Wilsdon, J., Matowe, L., Needham, G., Gilbert, F., et al. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04564-5 · Cited by 222 · see citing works
Self-citationSome Aspects of the Design and Analysis of Cluster Randomization Trials
Donner, A. P. (1998) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/1467-9876.00100 · Cited by 178 · see citing works
Matowe, L. K., Ramsay, C. R., Grimshaw, J., Gilbert, F. J., MacLeod, M. J., & Needham, G. (2002) · Clinical Radiology · doi:10.1053/crad.2001.0894 · Cited by 78 · see citing works
The miscarriage clinic: an audit of the first year
Turner, M. J., Flannelly, G., Wingfield, M. B., Rasmussen, M., Ryan, R., Cullen, S., Maguire, R., & STRONGE, J. M. (1991) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.1991.tb13398.x · Cited by 60 · see citing works
Economic and Medical Evaluation of Health Care Technologies
Culyer, A. J., & Horisberger, B. (1983) · Springer eBooks · doi:10.1007/978-3-642-69439-4 · Cited by 59 · see citing works
Lobo, C. M., Frijling, B. D., Hulscher, M., Bernsen, R. M. D., Braspenning, J. C. C., Grol, R. P. T. M., Prins, A., & van der Wouden, J. C. (2002) · Preventive Medicine · doi:10.1006/pmed.2002.1095 · Cited by 43 · see citing works
Mollison, J., Simpson, J. A., Campbell, M., & Grimshaw, J. (2000) · PubMed · Cited by 42 · see citing works
Randomization by Group in Studying the Effect of Drug Information in Primary Care
Diwan, V. K., Eriksson, B., Sterky, G., & Tomson, G. B. (1992) · International Journal of Epidemiology · doi:10.1093/ije/21.1.124 · Cited by 42 · see citing works
Computerized concurrent review of prenatal care.
Morgan, M., Studney, D. R., Go, B., & Winickoff, R. N. (1978) · PubMed · Cited by 41 · see citing works
Recruitment strategies in a cluster randomized trial – cost implications
Flynn, T. N., Whitley, E., & Peters, T. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1025 · Cited by 40 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Murray DM. The design and analysis of group randomised trials. Oxford: Oxford University Press, 1998.
- Cochran WG, Cox GM. Experimental design. New York: Wiley, 1957.
- Eccles M, McColl E, Steen N, et al. A cluster randomised controlled trial of computerised evidence based guidelines for angina and asthma in primary care. BMJ 2002;325:941-7.
- Matowe L, Ramsay C, Grimshaw JM, et al. Influence of the Royal College of Radiologists' guidelines on referrals from general practice: a time series analysis. Clin Radiol 2002;57:575-8.
- Grilli R, Ramsay CR, Minozi S. Mass media interventions: effects on health services utilisation In: Cochrane Collaboration. The Cochrane Library. Oxford: Update Software, 2002.
- Grilli R, Freemantle N, Minozzi S, et al. Impact of mass media on health services utilisation. In: Cochrane Collaboration. The Cochrane Library. Issue 3. Oxford: Update Software, 1998.
Cited by
Sorted by citations
Developing and Evaluating Complex Interventions: New Guidance
Craig, P., Dieppe, P. A., MacIntyre, S., Michie, S., Nazareth, I., & Petticrew, M. P. (2006) · UCL Discovery (University College London) · Cited by 1,877 · see citing works
Qualitative and Mixed Methods Provide Unique Contributions to Outcomes Research
Curry, L. A., Nembhard, I. M., & Bradley, E. H. (2009) · Circulation · doi:10.1161/circulationaha.107.742775 · Cited by 1,091 · see citing works
Local opinion leaders: effects on professional practice and health care outcomes
Flodgren, G., Parmelli, E., Doumit, G., Gattellari, M., O'Brien, M. A., Grimshaw, J., & Eccles, M. P. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000125.pub4 · Cited by 1,070 · see citing works
Self-citationA systematic review of primary prevention strategies for sexual violence perpetration
DeGue, S. A., Valle, L. A., Holt, M. K., Massetti, G. M., Matjasko, J. L., & Tharp, A. T. (2014) · Aggression and Violent Behavior · doi:10.1016/j.avb.2014.05.004 · Cited by 781 · see citing works
Interventions targeting social isolation in older people: a systematic review
Dickens, A. P., Richards, S. H., Greaves, C. J., & Campbell, J. L. (2011) · BMC Public Health · doi:10.1186/1471-2458-11-647 · Cited by 767 · see citing works
Gilbody, S., Whitty, P., Grimshaw, J., & Thomas, R. (2003) · JAMA · doi:10.1001/jama.289.23.3145 · Cited by 749 · see citing works
Evidence-Based Quality Improvement: The State Of The Science
Shojania, K. G., & Grimshaw, J. (2005) · Health Affairs · doi:10.1377/hlthaff.24.1.138 · Cited by 688 · see citing works
Patterns of intra-cluster correlation from primary care research to inform study design and analysis
Adams, G., Gulliford, M. C., Ukoumunne, O. C., Eldridge, S., Chinn, S., & Campbell, M. J. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.12.013 · Cited by 602 · see citing works
Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G. (2012) · Evaluation · doi:10.1177/1356389012442444 · Cited by 476 · see citing works
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015) · Implementation Science · doi:10.1186/s13012-014-0192-y · Cited by 435 · see citing works
Application of statistical process control in healthcare improvement: systematic review
Thor, J., Lundberg, J., Ask, J., Olsson, J., Carli, C., Härenstam, K. P., & Brommels, M. (2007) · BMJ Quality & Safety · doi:10.1136/qshc.2006.022194 · Cited by 432 · see citing works
Experimental and quasi-experimental designs in implementation research
Miller, C. J., Smith, S. N., & Pugatch, M. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2019.06.027 · Cited by 373 · see citing works
“widespread implementation of policy mandates, quality improvement initiatives, or dissemination campaigns (Eccles et al., 2003).”
BackgroundInfluentialNicolay, C., Purkayastha, S., Greenhalgh, A., Benn, J. J., Chaturvedi, S., Phillips, N., & Darzi, A. W. (2011) · British journal of surgery · doi:10.1002/bjs.7803 · Cited by 348 · see citing works
Clinical implications of congenital uterine anomalies: a meta-analysis of comparative studies
Venetis, C. A., Papadopoulos, S. P., Campo, R., Gordts, S., Tarlatzis, B. C., & Grimbizis, G. F. (2014) · Reproductive BioMedicine Online · doi:10.1016/j.rbmo.2014.09.006 · Cited by 332 · see citing works
Assessing risk of bias in a non‐randomized study
Sterne, J. A., Hernán, M. A., McAleenan, A., Reeves, B. C., & Higgins, J. P. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch25 · Cited by 330 · see citing works
How to study improvement interventions: a brief overview of possible study types
Portela, M. C., Pronovost, P. J., Woodcock, T. M., Carter, P., & Dixon‐Woods, M. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003620 · Cited by 318 · see citing works
Systematic reviews of the effectiveness of quality improvement strategies and programmes
Grimshaw, J. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.4.298 · Cited by 290 · see citing works
Hawe, P., Shiell, A., Riley, T., & Gold, L. H. (2004) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2003.014415 · Cited by 287 · see citing works
Achieving Rapid Door-To-Balloon Times
Bradley, E. H., Curry, L. A., Webster, T. R., Mattera, J. A., Roumanis, S. A., Radford, M. J., McNamara, R. L., Barton, B. A., Berg, D. N., & Krumholz, H. M. (2006) · Circulation · doi:10.1161/circulationaha.105.590133 · Cited by 279 · see citing works
De-implementing wisely: developing the evidence base to reduce low-value care
Grimshaw, J., Patey, A. M., Kirkham, K. R., Hall, A. M., Dowling, S. K., Rodondi, N., Ellen, M., Kool, R. B., van Dulmen, S. A., Kerr, E. A., et al. (2020) · BMJ Quality & Safety · doi:10.1136/bmjqs-2019-010060 · Cited by 278 · see citing works
Campbell, M. K., Fayers, P. M., & Grimshaw, J. M. (2005) · Clinical Trials · doi:10.1191/1740774505cn071oa · Cited by 275 · see citing works
Trogrlić, Z., van der Jagt, M., Bakker, J., Balas, M. C., Ely, E. W., van der Voort, P. H. J., & Ista, E. (2015) · Critical Care · doi:10.1186/s13054-015-0886-9 · Cited by 271 · see citing works
Spall, H. G. C. V., Lee, S. F., Xie, F., Oz, U. E., Perez, R., Mitoff, P. R., Maingi, M., Tjandrawidjaja, M. C., Heffernan, M., Zia, M. I., et al. (2019) · JAMA · doi:10.1001/jama.2019.0710 · Cited by 264 · see citing works
Study Designs for Effectiveness and Translation Research
Mercer, S. L., DeVinney, B. J., Fine, L. J., Green, L. W., & Dougherty, D. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.04.005 · Cited by 222 · see citing works
Rehabilitation Interventions for Postintensive Care Syndrome
Mehlhorn, J., Freytag, A., Schmidt, K. F. R., Brunkhorst, F. M., Graf, J., Troitzsch, U., Schlattmann, P., Wensing, M., & Gensichen, J. S. (2014) · Critical Care Medicine · doi:10.1097/ccm.0000000000000148 · Cited by 222 · see citing works
Evaluating the impact of prediction models: lessons learned, challenges, and recommendations
Kappen, T. H., van Klei, W. A., van Wolfswinkel, L., Kalkman, C. J., Vergouwe, Y., & Moons, K. G. M. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0033-6 · Cited by 215 · see citing works
Local opinion leaders: effects on professional practice and health care outcomes
Flodgren, G., Parmelli, E., Doumit, G. D., Gattellari, M., O’Brien, M. A., Grimshaw, J., & Eccles, M. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000125.pub3 · Cited by 191 · see citing works
Self-citationvan der Veer, S. N., de Keizer, N. F., Ravelli, A. C. J., Tenkink, S., & Jager, K. J. (2010) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2010.01.011 · Cited by 184 · see citing works
Blot, K. Y., Bergs, J., Vogelaers, D., Blot, S. I., & Vandijck, D. M. (2014) · Clinical Infectious Diseases · doi:10.1093/cid/ciu239 · Cited by 184 · see citing works
Using clinical simulation to study how to improve quality and safety in healthcare
Lamé, G., & Dixon‐Woods, M. (2018) · BMJ Simulation & Technology Enhanced Learning · doi:10.1136/bmjstel-2018-000370 · Cited by 180 · see citing works
Realist RCTs of complex interventions – An oxymoron
Marchal, B., Westhorp, G., Wong, G., Belle, S. V., Greenhalgh, T., Kegels, G., & Pawson, R. (2013) · Social Science & Medicine · doi:10.1016/j.socscimed.2013.06.025 · Cited by 168 · see citing works
What Is Implementation Research?
Bhattacharyya, O. K., Reeves, S., & Zwarenstein, M. (2009) · Research on Social Work Practice · doi:10.1177/1049731509335528 · Cited by 162 · see citing works
Primary care models for treating opioid use disorders: What actually works? A systematic review
Lagisetty, P., Klasa, K., Bush, C. A., Heisler, M., Chopra, V., & Bohnert, A. S. B. (2017) · PLoS ONE · doi:10.1371/journal.pone.0186315 · Cited by 155 · see citing works
Models, Strategies, and Tools.
Sales, A. E., Smith, J. L., Curran, G. M., & Kochevar, L. K. (2006) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2006.00362.x · Cited by 150 · see citing works
Pinnock, H., Epiphaniou, E., Sheikh, A., Griffiths, C., Eldridge, S., Craig, P., & Taylor, S. J. (2015) · Implementation Science · doi:10.1186/s13012-015-0235-z · Cited by 128 · see citing works
Huis, A. M. P., Schoonhoven, L., Grol, R. P. T. M., Donders, R., Hulscher, M., & van Achterberg, T. (2012) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2012.08.004 · Cited by 143 · see citing works
Robison, J., Curry, L. C., Gruman, C., Porter, M., Henderson, C., & Pillemer, K. A. (2007) · The Gerontologist · doi:10.1093/geront/47.4.504 · Cited by 139 · see citing works
Knowledge Translation Research: The Science of Moving Research Into Policy and Practice
Curran, J., Grimshaw, J., Hayden, J. A., & Campbell, B. R. (2011) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.20124 · Cited by 135 · see citing works
How to Use an Article About Quality Improvement
Fan, E., Laupacis, A., Pronovost, P. J., Guyatt, G. H., & Needham, D. M. (2010) · JAMA · doi:10.1001/jama.2010.1692 · Cited by 135 · see citing works
Hanson, D., Finch, C. F., Allegrante, J. P., & Sleet, D. A. (2012) · Public Health Reports · doi:10.1177/003335491212700203 · Cited by 127 · see citing works
Educational interventions designed to develop nurses’ cultural competence: A systematic review
Oikarainen, A., Mikkonen, K., Kenny, A. J., Tomietto, M., Tuomikoski, A., Meriläinen, M. H., Miettunen, J., & Kääriäinen, M. (2019) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2019.06.005 · Cited by 124 · see citing works
Systematic review of public-targeted communication interventions to improve antibiotic use
Cross, E. L. A., Tolfree, R., & Kipping, R. (2017) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkw520 · Cited by 118 · see citing works
What are effective strategies for the implementation of care bundles on ICUs: a systematic review
Borgert, M. J., Goossens, A., & Dongelmans, D. A. (2015) · Implementation Science · doi:10.1186/s13012-015-0306-1 · Cited by 116 · see citing works
The role of organizational research in implementing evidence-based practice: QUERI Series
Yano, E. M. (2008) · Implementation Science · doi:10.1186/1748-5908-3-29 · Cited by 112 · see citing works
Wells, S., Furness, S., Rafter, N., Horn, E., Whittaker, R., Stewart, A., Moodabe, K., Roseman, P., Selak, V., Bramley, D., et al. (2008) · European Journal of Cardiovascular Prevention & Rehabilitation · doi:10.1097/hjr.0b013e3282f13af4 · Cited by 109 · see citing works
Berdot, S., Roudot, M., Schramm, C., Katsahian, S., Durieux, P., & Sabatier, B. (2015) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2015.08.012 · Cited by 110 · see citing works
Shanbhag, D., Graham, I. D., Harlos, K., Haynes, R. B., Gabizon, I., Connolly, S. J., & Spall, H. G. C. V. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-017765 · Cited by 105 · see citing works
A proposed method of bias adjustment for meta-analyses of published observational studies
Thompson, S., Ekelund, U., Jebb, S., Lindroos, A. K., Mander, A., Sharp, S., Turner, R., & Wilks, D. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyq248 · Cited by 101 · see citing works
Bastuji‐Garin, S., Sbidian, É., Gaudy‐Marqueste, C., Ferrat, É., Roujeau, J., Richard, M. A., Canouï‐Poitrine, F., & on behalf of the European Dermatology Network (EDEN) (2013) · PLoS ONE · doi:10.1371/journal.pone.0064733 · Cited by 101 · see citing works
Secular trends and evaluation of complex interventions: the rising tide phenomenon
Chen, Y., Hemming, K., Stevens, A. J., & Lilford, R. J. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004372 · Cited by 102 · see citing works
Howells, K., Sivaratnam, C. S., May, T., Lindor, E., McGillivray, J., & Rinehart, N. (2019) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-019-04050-9 · Cited by 99 · see citing works
Backman, C., Zoutman, D., & Marck, P. (2008) · American Journal of Infection Control · doi:10.1016/j.ajic.2007.08.007 · Cited by 99 · see citing works
Soft regulations in pharmaceutical policy making
Wettermark, B., Godman, B., Jacobsson, B., & Haaijer-Ruskamp, F. (2009) · Applied Health Economics and Health Policy · doi:10.1007/bf03256147 · Cited by 101 · see citing works
Rashoud, A. S. A., Abboud, R., Wang, W., & Wigderowitz, C. A. (2013) · Physiotherapy · doi:10.1016/j.physio.2013.09.007 · Cited by 96 · see citing works
Self, W. H., Speroff, T., Grijalva, C. G., McNaughton, C. D., Ashburn, J., Liu, D., Arbogast, P. G., Russ, S., Storrow, A. B., & Talbot, T. R. (2013) · Academic Emergency Medicine · doi:10.1111/acem.12057 · Cited by 92 · see citing works
Interventions for reducing medication errors in children in hospital
Maaskant, J. M., Vermeulen, H., Apampa, B., Fernando, B., Ghaleb, M. A., Neubert, A. C., Thayyil, S., & Soe, A. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006208.pub3 · Cited by 92 · see citing works
de Kraker, M. E. A., Abbas, M., Huttner, B. D., & Harbarth, S. (2017) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2017.05.019 · Cited by 91 · see citing works
Evans, C., Harding, R., Higginson, I. J., & MORECare, O. B. O. (2013) · Palliative Medicine · doi:10.1177/0269216312467489 · Cited by 91 · see citing works
Review: What Can We Learn From Quality Improvement Research?
Alexander, J. A., & Hearld, L. R. (2009) · Medical Care Research and Review · doi:10.1177/1077558708330424 · Cited by 91 · see citing works
Jibb, L., Nathan, P. C., Stevens, B., Seto, E., Cafazzo, J. A., Stephens, N., Yohannes, L., & Stinson, J. (2015) · Oncology nursing forum · doi:10.1188/15.onf.e339-e357 · Cited by 91 · see citing works
Wells, K. B., Staunton, A., Norris, K. C., Bluthenthal, R. N., Chung, B., Gelberg, L., Jones, L., Kataoka, S., Koegel, P., Miranda, J. M., et al. (2006) · PubMed · Cited by 89 · see citing works
Barnett, P., Matthews, H., Lloyd-Evans, B., Mackay, E., Pilling, S., & Johnson, S. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30382-1 · Cited by 89 · see citing works
Lin, Y., Lee, W. C., Kuo, L., Cheng, Y.-C., Lin, C., Lin, H.-L., Chen, C.-W., & Lin, T. (2013) · BMC Medical Ethics · doi:10.1186/1472-6939-14-8 · Cited by 87 · see citing works
Werfalli, M., Raubenheimer, P. J., Engel, M. E., Musekiwa, A., Bobrow, K., Peer, N., Hoegfeldt, C. A., Kalula, S., Kengne, A. P., & Levitt, N. S. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01377-8 · Cited by 93 · see citing works
Patient Navigation: Development of a Protocol for Describing What Navigators Do
Parker, V. A., Clark, J. A., Leyson, J., Calhoun, E. A., Carroll, J. K., Freund, K. M., & Battaglia, T. A. (2010) · Health Services Research · doi:10.1111/j.1475-6773.2009.01079.x · Cited by 85 · see citing works
Amaratunga, T., & Dobranowski, J. (2016) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2016.02.033 · Cited by 83 · see citing works
The demonstration of a theory-based approach to the design of localized patient safety interventions
Taylor, N., Lawton, R., Slater, B., & Foy, R. (2013) · Implementation Science · doi:10.1186/1748-5908-8-123 · Cited by 77 · see citing works
Interrupted Time-Series Analysis of Regulations to Reduce Paracetamol (Acetaminophen) Poisoning
Morgan, O. W., Griffiths, C., & Majeed, A. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040105 · Cited by 82 · see citing works
Godeau, E., Vignes, C., Sentenac, M., Ehlinger, V., NAVARRO, F., Grandjean, H., & Arnaud, C. (2010) · Developmental Medicine & Child Neurology · doi:10.1111/j.1469-8749.2010.03731.x · Cited by 79 · see citing works
Raab, S. S., Andrew-Jaja, C., Condel, J. L., & Dabbs, D. J. (2006) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2005.06.069 · Cited by 75 · see citing works
Goud, R., de Keizer, N. F., ter Riet, G., Wyatt, J. C., Hasman, A., Hellemans, I. M., & Peek, N. B. (2009) · BMJ · doi:10.1136/bmj.b1440 · Cited by 76 · see citing works
Borgestig, M., Sandqvist, J. L., Ahlsten, G., Falkmer, T., & Hemmingsson, H. (2016) · Developmental Neurorehabilitation · doi:10.3109/17518423.2015.1132281 · Cited by 76 · see citing works
An examination of the rigor and value of final scholarly projects completed by DNP nursing students
Roush, K. M., & Tesoro, M. G. (2018) · Journal of Professional Nursing · doi:10.1016/j.profnurs.2018.03.003 · Cited by 75 · see citing works
Perioperative care bundles for the prevention of surgical-site infections: meta-analysis
Wolfhagen, N., Boldingh, Q. J. J., Boermeester, M. A., & de Jonge, S. W. (2022) · British journal of surgery · doi:10.1093/bjs/znac196 · Cited by 79 · see citing works
Vázquez, M. L., Vargas, Í., Unger, J., Paepe, P. D., Mogollón‐Pérez, A., Samico, I., de Albuquerque, P. C., Eguiguren, P., Cisneros, A.-I., Rovere, M., et al. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-007037 · Cited by 74 · see citing works
Ventilator Bundle and Its Effects on Mortality Among ICU Patients: A Meta-Analysis*
Pileggi, C., Mascaro, V., Bianco, A., Nobile, C. G. A., & Pavía, M. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003136 · Cited by 74 · see citing works
The effectiveness of regionalization of perinatal care services - a systematic review
Rashidian, A., Omidvari, A., Vali, Y., Mortaz, S., Nooraie, R. Y., Jafarı, M., & Bhutta, Z. A. (2014) · Public Health · doi:10.1016/j.puhe.2014.08.005 · Cited by 72 · see citing works
Rooming-in for Infants at Risk of Neonatal Abstinence Syndrome
McKnight, S., Coo, H., Davies, G., Holmes, B., Newman, A., Newton, L., & Dow, K. E. (2015) · American Journal of Perinatology · doi:10.1055/s-0035-1566295 · Cited by 71 · see citing works
Charlton, K., Comerford, T., Deavin, N., & Walton, K. (2021) · Public Health Nutrition · doi:10.1017/s1368980020004024 · Cited by 90 · see citing works
Visschere, L. D., Schols, J. M. G. A., van der Putten, G., Baat, C. D., & Vanobbergen, J. (2010) · Gerodontology · doi:10.1111/j.1741-2358.2010.00418.x · Cited by 70 · see citing works
Berg, R. C., & Denison, E. (2012) · Studies in Family Planning · doi:10.1111/j.1728-4465.2012.00311.x · Cited by 70 · see citing works
Improving the use of research evidence in guideline development: 16. Evaluation
Oxman, A. D., Schünemann, H. J., & Fretheim, A. (2006) · Health Research Policy and Systems · doi:10.1186/1478-4505-4-28 · Cited by 70 · see citing works
Moja, L., Friz, H. P., Capobussi, M., Kwag, K. H., Banzi, R., Ruggiero, F. M., González‐Lorenzo, M., Liberati, E. G., Mangia, M., Nyberg, P. M., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.17094 · Cited by 68 · see citing works
van Steenkiste, B., van der Weijden, T., Stoffers, J., Kester, A. D. M., Timmermans, D. R. M., & Grol, R. P. T. M. (2007) · European Journal of Cardiovascular Prevention & Rehabilitation · doi:10.1097/01.hjr.0000239475.71805.1e · Cited by 69 · see citing works
Effectiveness studies: advantages and disadvantages
Möller, H. (2011) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2011.13.2/hmoeller · Cited by 69 · see citing works
holt, A. F. V. I. ’., Helder, O. K., Vos, M. D., Schafthuizen, L., Sülz, S., van den Hoogen, A., & Ista, E. (2017) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2017.01.007 · Cited by 68 · see citing works
Siu, G., Wight, D., Seeley, J., Namutebi, C., Sekiwunga, R., Zalwango, F., & Kasule, S. (2017) · European Journal of Development Research · doi:10.1057/s41287-017-0103-6 · Cited by 67 · see citing works
Foy, R., Francis, J. J., Johnston, M., Eccles, M., Lecouturier, J., Bamford, C., & Grimshaw, J. (2007) · BMC Health Services Research · doi:10.1186/1472-6963-7-207 · Cited by 66 · see citing works
Self-citationVarallo, F. R., Planeta, C. S., & Mastroianni, P. D. C. (2017) · Clinics · doi:10.6061/clinics/2017(01)09 · Cited by 66 · see citing works
Gonzales, R., Corbett, K. K., Wong, S. L., Glazner, J. E., Deas, A. M., Leeman-Castillo, B. A., Maselli, J. H., Sebert-Kuhlmann, A. K., Wigton, R. S., Flores, E. T., et al. (2008) · Medical Care · doi:10.1097/mlr.0b013e3181653d2e · Cited by 66 · see citing works
Charting a course for health services research at the National Institute on Drug Abuse
Compton, W. M., Stein, J. B., Robertson, E. B., Pintello, D., Pringle, B., & Volkow, N. D. (2005) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2005.05.008 · Cited by 65 · see citing works
Methods for evaluation of small scale quality improvement projects
Harvey, G. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.3.210 · Cited by 65 · see citing works
Donaldson, M. S. (2008) · Quality of Life Research · doi:10.1007/s11136-008-9414-6 · Cited by 64 · see citing works
Bhattacharyya, O. K., Estey, E. A., & Zwarenstein, M. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.022 · Cited by 64 · see citing works
Iliffe, S., Wilcock, J., Drennan, V. M., Goodman, C. M., Griffin, M., Knapp, M. R. J., Lowery, D., Manthorpe, J., Rait, G., & Warner, J. P. (2015) · Programme Grants for Applied Research · doi:10.3310/pgfar03030 · Cited by 63 · see citing works
Evaluating medical tourism enablers with interpretive structural modeling
Debata, B. R., Sreekumar, S., Patnaik, B., & Mahapatra, S. S. (2013) · Benchmarking An International Journal · doi:10.1108/bij-10-2011-0079 · Cited by 63 · see citing works
Gagliardi, A. R., Fenech, D. S., Eskicioglu, C., Nathens, A. B., & McLeod, R. S. (2009) · Europe PMC (PubMed Central) · Cited by 63 · see citing works
Hutton, J. L., Eccles, M., & Grimshaw, J. (2008) · Implementation Science · doi:10.1186/1748-5908-3-52 · Cited by 62 · see citing works
Self-citationThe value of monitoring frozen section-permanent section correlation data over time.
Raab, S. S., Tworek, J. A., Souers, R. J., & Zarbo, R. J. (2006) · PubMed · doi:10.5858/2006-130-337-tvomfs · Cited by 50 · see citing works
Impact on hospital admissions of an integrated primary care model for very frail elderly patients
de Stampa, M., Vedel, I., Buyck, J.-F., Lapointe, L., Bergman, H., Béland, F., & Ankri, J. (2014) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2014.01.005 · Cited by 60 · see citing works
- Authors
- Eccles, M.
- Affiliations
- Newcastle University (GB)
- Corresponding author
- Eccles, M
- ORCID
- Eccles, M
- Venue
- Quality and Safety in Health Care
- Publisher
- BMJ
- Published
- 2003-02-01
- Volume
- 12
- Issue
- 1
- Pages
- 47-52
- ISSN
- 1475-3898 (ISSN-L)
- 1475-3901
- PMC
- PMC1743658
- OpenAlex
- W2046947214
- Cited by
- 586
- References
- 24
- Citations by year
- 2026: 8 · 2025: 11 · 2024: 16 · 2023: 21 · 2022: 16 · 2021: 24 · 2020: 34 · 2019: 44 · 2018: 31 · 2017: 40 · 2016: 35 · 2015: 44 · 2014: 32 · 2013: 24 · 2012: 23
- FWCI
- 35.41
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- other-oa
- Locations
- BMJ Quality & Safety (publishedVersion, other-oa)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Newcastle University
- University of Aberdeen
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
- Keywords
- research design, improvement measures, randomized controlled trials, factorial design, block designs, before-after study, time series analysis, bias, generalizability, intervention development
- MeSH terms
- United Kingdom, Health Services Research, Humans, Organizational Innovation, Research Design, State Medicine, Reproducibility of Results, Program Evaluation, Cluster Analysis, Randomized Controlled Trials as Topic, Total Quality Management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
