The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Research designs for studies evaluating the effectiveness of change and improvement strategies
Open accessAbstract
The methods of evaluating change and improvement strategies are not well described. The design and conduct of a range of experimental and non-experimental quantitative designs are considered. Such study designs should usually be used in a context where they build on appropriate theoretical, qualitative and modelling work, particularly in the development of appropriate interventions. A range of experimental designs are discussed including single and multiple arm randomised controlled trials and the use of more complex factorial and block designs. The impact of randomisation at both group and individual levels and three non-experimental designs (uncontrolled before and after, controlled before and after, and time series analysis) are also considered. The design chosen will reflect both the needs (and resources) in any particular circumstances and also the purpose of the evaluation. The general principle underlying the choice of evaluative design is, however, simple-those conducting such evaluations should use the most robust design possible to minimise bias and maximise generalisability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The interventions may have been poorly developed, lacking appropriate theoretical basis and subsequent components to impact on the measured outcome domains [82,83].”
From Interventions targeting social isolation in older people: a systematic review (2011) · cites this paper as [82]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1963) · Medical Entomology and Zoology · Cited by 15,085
(1979) · Medical Entomology and Zoology · Cited by 11,252
Quasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,091
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474
Clinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,918
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033
Effectiveness and Efficiency: Random Reflections on Health Services
L'Etang, H. (1973) · Public Health · doi:10.1016/s0033-3506(73)80082-4 · Cited by 1,905
Explanatory and Pragmatic Attitudes in Therapeutical Trials
Schwartz, D., & Lellouch, J. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.01.012 · Cited by 1,253
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236
Time-Series Analysis: Regression Techniques.
Pampel, F. C., & Ostrom, C. W. (1980) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2065616 · Cited by 499
Flagged first, then by citations
Research works printed but not linked: 6
- Murray DM. The design and analysis of group randomised trials. Oxford: Oxford University Press, 1998.
- Cochran WG, Cox GM. Experimental design. New York: Wiley, 1957.
- Eccles M, McColl E, Steen N, et al. A cluster randomised controlled trial of computerised evidence based guidelines for angina and asthma in primary care. BMJ 2002;325:941-7.
- Matowe L, Ramsay C, Grimshaw JM, et al. Influence of the Royal College of Radiologists' guidelines on referrals from general practice: a time series analysis. Clin Radiol 2002;57:575-8.
- Grilli R, Ramsay CR, Minozi S. Mass media interventions: effects on health services utilisation In: Cochrane Collaboration. The Cochrane Library. Oxford: Update Software, 2002.
- Grilli R, Freemantle N, Minozzi S, et al. Impact of mass media on health services utilisation. In: Cochrane Collaboration. The Cochrane Library. Issue 3. Oxford: Update Software, 1998.
Cited by
Sorted by citations
Items 501–585 of 585
Aboriginal and Torres Strait Islander women: an examination of smoking during pregnancy
Gilligan, C. (2008) · NOVA (University of Newcastle Australia) · Cited by 1
Carey, M. L., & Sanson‐Fisher, R. (2011) · Oxford University Press eBooks · doi:10.1093/obo/9780199756797-0048 · Cited by 1
Experimental Designs for Evaluation of Implementation Strategies
Wensing, M., & Grimshaw, J. (2020) · doi:10.1002/9781119488620.ch20 · Cited by 1
de Rezende, H. A. (2020) · doi:10.11606/t.7.2020.tde-24022021-155624 · Cited by 1
Foundations of Scholarly WritingPreprint
Heston, T. F. (2023) · doi:10.22541/au.170216752.26604161/v1 · Cited by 1
Setting- und Kontexteinflüsse in evidenzbasierten Leitlinien
Muth, C., Beyer, M., Gensichen, J. S., & Gerlach, F. M. (2009) · Springer eBooks · doi:10.1007/978-3-540-73042-2_14
BALANCE AND VESTIBULAR REHABILITATION QUALITY IMPROVEMENT
ALMohiza, M. A. (2014) · D-Scholarship@Pitt (University of Pittsburgh)
Dirks, M. (2005) · RePub (Erasmus University Rotterdam)
van der Weijden, T., & Wensing, M. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat03997
Validade preditiva de métodos para a avaliação da atividade das lesões de cárie em dentes decíduos
Martins, I. F. N. (2014) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/d.23.2014.tde-11072014-152800
Drills and Exercises as Interventions to Improve Public Health Emergency Response
Knutson, D. B. (2011) · ScholarWorks (Walden University)
van Steenkiste, B. (2008) · Research Publications (Maastricht University) · doi:10.26481/dis.20080619bs
Martens, J. D. (2007) · Research Publications (Maastricht University) · doi:10.26481/dis.20071107jm
Impact d’une fiche de liaison relative aux soins de support à destination des médecins traitants
Fouquereau, J., Laval, G., & Labarère, J. (2011) · Bulletin du Cancer · doi:10.1684/bdc.2011.1452
Evidenzbasierung und leitliniengestützte Therapie in der Psychiatrie
Möller, H. (2011) · Springer eBooks · doi:10.1007/978-3-642-03637-8_43
Cleland, C. M. (2012) · Substance Use & Misuse · doi:10.3109/10826084.2012.723490
Studying the Effectiveness of Palliative Care—Reply
Zimmermann, C. (2008) · JAMA · doi:10.1001/jama.300.9.1023
Clinical practice guidelines in Portugal : quantitative and qualitative evaluation
Rodrigues, D. M. S. (2015) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Nyman, V. (2015) · Gothenburg University Publications Electronic Archive (Gothenburg University)
Intervention Research in Health Care
Sobolev, B. G., Sánchez, V., & Kuramoto, L. (2012) · Springer eBooks · doi:10.1007/978-1-4614-2233-4_1
A Multi-Level Model of Personal Victimization Among South Korean Youths
Cho, S. (2015) · OhioLink ETD Center (Ohio Library and Information Network)
Poon, T., & Kritek, P. (2016) · Respiratory Care · doi:10.4187/respcare.05013
Community-based and Innovative Technological Approaches to Improve Child Nutrition in India
Singh, N. (2015) · LSHTM Research Online (London School of Hygiene and Tropical Medicine) · doi:10.17037/pubs.02030984
7 Wetenschappelijk onderzoek naar kwaliteitsverbetering in de patiëntenzorg
Wollersheim, H. C. H., & van der Weijden, T. (2011) · Bohn Stafleu van Loghum eBooks · doi:10.1007/978-90-313-8236-1_7
Translation Research and Major Ongoing Clinical Trials
Benjamin, E. M. (2010) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195317060.003.0030
Students performance in Face to Face vs Blended Learning
References, B., Morais, A., SILVA, A. C. G., Gulbenkian, C., Melli, N., Moscati, S., Neto, J. D. D. O., de Araújo, U. F., Marques, G. R. T. D., Clark, R., et al. (2016)
Improving diabetes care in Aboriginal communities in the Northern Territory
Si, D. (2006) · Charles Darwin University · doi:10.25913/5eaa17e184d9c
Bravo-Peña, M., Barona-Fong, L., Campo-López, J., Arroyave, Y., & Calvache, J. A. (2016) · Colombian Journal of Anesthesiology · doi:10.1016/j.rca.2016.09.005
Maznytsya, A. (2016) · TSpace (University of Toronto)
Takeuchi, E. E. (2016) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York)
Picture this - Mobile photo radar and its efficacy on traffic safety
Gariepy, J. (2017) · UVic’s Research and Learning Repository (University of Victoria)
Research in the sciences of improvement, implementation, and pediatric patient safety
Lachman, P. I. (2017) · Pediatric Research · doi:10.1038/pr.2017.282
Yellur, R., & Hassan, S. (2017) · doi:10.7287/peerj.preprints.3413v1
Labarère, J., Belle, L., Fourny, M., Vanzetto, G., Debaty, G. P., Delgado‐Gómez, D., Brallet, J., Vallet, B., & Danchin, N. (2012)
Morganheira, D., Silva, P., Pereira, R., & Ruivo, A. (2018) · Revista Ibero-Americana de Saúde e Envelhecimento · doi:10.24902/r.riase.2017.3(2).1000
Logan, G. (2018) · Journal of research in nursing · doi:10.1177/1744987118756782
Martins, I. F. N. (2018) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.23.2018.tde-27112017-121756
Towards better management of obstructive sleep apnoea in tetraplegia
Graco, M. (2018) · Minerva Access (University of Melbourne)
Laur, C. V. (2019) · UWSpace (University of Waterloo)
Quality Improvement Initiative for Assessing Allografts after Lung Transplantation
Hayes, D., Nicholson, K. L., Miller, R., Nance, A. E., Kirkby, S. E., Parker, W. F., & Baker, P. B. (2019) · Pediatric Quality and Safety · doi:10.1097/pq9.0000000000000146
Foster, M. (2019) · uO Research (University of Ottawa) · doi:10.20381/ruor-23416
Engaging Internal Medicine Hospitalists in Antimicrobial Stewardship—a Systematic Review
Brooks, M. N., & Stevens, M. (2020) · Current treatment options in infectious diseases/Current treatment options in infectious disease · doi:10.1007/s40506-020-00226-5
Evaluatief onderzoek naar kwaliteitsverbetering in de patiëntenzorg
Wollersheim, H. C. H., & van der Weijden, T. (2020) · Bohn Stafleu van Loghum eBooks · doi:10.1007/978-90-368-2360-9_7
Health services research in palliative and end of life care
Smets, T., & Deliëns, L. (2021) · doi:10.1093/med/9780198821328.003.0126
Evidenzbasierung und leitliniengestützte Therapie in der Psychiatrie
Leucht, S., & Möller, H. (2017) · Springer eBooks · doi:10.1007/978-3-662-49295-6_55
Evidenzbasierung und leitliniengestützte Therapie in der Psychiatrie
Möller, H. (2007) · Springer eBooks · doi:10.1007/978-3-540-33129-2_41
van der Weijden, T., & Wensing, M. (2007) · Encyclopedia of Statistics in Quality and Reliability · doi:10.1002/9780470061572.eqr332
Bravo-Peña, M., Barona-Fong, L., Campo-López, J., Arroyave, Y., & Calvache, J. A. (2017) · Colombian Journal of Anesthesiology · doi:10.1097/01819236-201701000-00006
Evidenzbasierung und leitliniengeschützte Therapie in der Psychiatrie
Leucht, S., & Möller, H. (2016) · Springer eBooks · doi:10.1007/978-3-642-45028-0_55-1
Greenhalgh, T., Robert, G., Bate, P., Macfarlane, F., & Kyriakidou, O. (2005) · Wiley eBooks · doi:10.1002/9780470987407.refs
Implementación de GPC en el SNS. Manual metodológico
(2009) · GuíaSalud eBooks · doi:10.46995/man_6
Effecten van implementatie: het experimentele studiedesign
Wensing, M., & Grol, R. P. T. M. (2023) · Bohn Stafleu van Loghum eBooks · doi:10.1007/978-90-368-2909-0_20
Grimshaw, J., & Patey, A. M. (2023) · doi:10.1002/9781119862758.ch4
Indeed, Correlation Does Not Indicate Causation!
Nunes, G. S., de Moraes, W. S. L. A., Sampaio, V. D. S., Seda, N. R., Mouta, G. D. S., Dangui, A. J. M., Petersen, R. D. S., & Nakagawa, T. H. (2023) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2023.0203-r
Resticka, G. A., Sulistyaningtyas, S., Polando, J., Riski, F., Megawarni, Y., Muhammad Fachri, A. D., Juwanda, P. S., Serlyna, S., Lestari, A., & Ulmi, M. R. (2023) · Pamasa Jurnal Pengabdian Pada Masyarakat · doi:10.20884/1.pamasa.2023.1.2.8861
Kenis, I., Hecke, A. V., & Foulon, V. (2024) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.14027
Pontes, L. C. D. F., Pimenta, I. D. S. F., de Medeiros, G. C. B. S., Martínez, D. G., Pérez, P. E., Piuvezam, G., & Capucho, H. C. (2025) · PLoS ONE · doi:10.1371/journal.pone.0314201
Freedman, B. A. (2019) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc1538794
Smith, C. C., Caleshu, C. A., Bonadies, D. C., & Denton, J. J. (2025) · Journal of Genetic Counseling · doi:10.1002/jgc4.70047
Bornemann, G., Dringenberg, L., & Beer, U.-M. (2025) · Prävention und Gesundheitsförderung · doi:10.1007/s11553-025-01244-1
Ravita, R., Suman, S., Kundu, R., & Sharma, S. (2025) · Research Square · doi:10.21203/rs.3.rs-7158310/v1
Nosamudiana, D. O., Adandom, I. I., Fawole, H. O., Rayner, D. G., Ehigie, B., Ohuoba, S., Okoro, S. I., & Kalu, M. E. (2025) · Educational Gerontology · doi:10.1080/03601277.2025.2582561
“Quality” in Headache Care: What Is It and How Can It Be Measured?
Peters, M., Jenkinson, C., Perera, S., Loder, E., & Steiner, T. J. (2011) · Springer eBooks · doi:10.1007/978-88-470-1700-9_6
Sieleunou, I., & Bonong, P. R. E. (2024) · Figshare · doi:10.6084/m9.figshare.26718066
Blackaby, J., Byrne, J., Bellass, S., Canvin, K., & Foy, R. (2024) · Figshare · doi:10.6084/m9.figshare.26558792
Foy, R., Francis, J. J., Johnston, M., Eccles, M., Lecouturier, J., Bamford, C. H., & Grimshaw, J. (2021) · Figshare · doi:10.6084/m9.figshare.15079595
Self-citationAlias, S. B., Monteserín Nadal, R., Moral, I., Figuls, M. R. I., Luque, X. R. I., & Coll‐Planas, L. (2021) · Figshare · doi:10.6084/m9.figshare.13544471
Blackaby, J., Byrne, J., Bellass, S., Canvin, K., & Foy, R. (2024) · Figshare · doi:10.6084/m9.figshare.26558795
Berdot, S., Vilfaillot, A., Bézie, Y., Perrin, G., Berge, M., Corny, J., Thi, T. T. P., Depoisson, M., Guihaire, C., Valin, N., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16435821
Abd-Rouf, M. S. K., Faniyi, A. J., Abd-Rouf, A. B., Oladapo, B. G., Oladapo, O., Amusat, S. O., Jamiu, A. B., & Lucas, C. (2025) · Journal of Radiation and Cancer Research · doi:10.4103/jrcr.jrcr_5_25
Kim, M. J., Prieto‐Merino, D., Nicholas, J. M., Allen, L., Bastawrous, A., & Macleod, D. (2026) · BMC Medical Research Methodology · doi:10.1186/s12874-026-02780-w
Organizational impact in healthcare in France: a decade of insights but chicken and egg situation
Martin, T., Garces, G. A., Roussel, C., Martelli, N., & Marcellin, O. (2026) · International Journal of Technology Assessment in Health Care · doi:10.1017/s026646232610347x
How might we interpret quantitative findings in simulation research more meaningfully?
Schram, A. (2026) · Advances in Simulation · doi:10.1186/s41077-026-00437-8
Acute Kidney Injury Prevention in the Real World: Is Bundled Approach a Scalable Solution?
Sodhi, K., & Sahni, N. (2026) · Indian Journal of Critical Care Medicine · doi:10.5005/jp-journals-10071-25248
Birkeli, G. H., Thomas, O. M. T., Ballangrud, R., Deilkås, E. C. T., & Lindahl, A. K. (2026) · BMC Health Services Research · doi:10.1186/s12913-026-15405-1
Assessing the feasibility of an education intervention for triage interruption management
Johnson, K. D., Wagner, R., Guan, T.-Y., Barbosa, S. D. F. F., & Sasangohar, F. (2026) · Acta Paulista de Enfermagem · doi:10.37689/acta-ape/2026ao2013i
Maloy, C. (2005) · UKnowledge (University of Kentucky)
Yeats, R. M. (2013) · University of Birmingham Institutional Research Archive (University of Birmingham)
Metadata
- Authors
- Eccles, M.
- Affiliations
- Newcastle University (GB)
- Corresponding author
- Eccles, M
- ORCID
- Eccles, M
- Venue
- Quality and Safety in Health Care
- Publisher
- BMJ
- Published
- 2003-02-01
- Volume
- 12
- Issue
- 1
- Pages
- 47-52
- ISSN
- 1475-3898 (ISSN-L)
- 1475-3901
- PMC
- PMC1743658
- OpenAlex
- W2046947214
- Cited by
- 586
- References
- 24
- Citations by year
- 2026: 8 · 2025: 11 · 2024: 16 · 2023: 21 · 2022: 16 · 2021: 24 · 2020: 34 · 2019: 44 · 2018: 31 · 2017: 40 · 2016: 35 · 2015: 44 · 2014: 32 · 2013: 24 · 2012: 23
- FWCI
- 35.41
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- other-oa
- Locations
- BMJ Quality & Safety (publishedVersion, other-oa)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Newcastle University
- University of Aberdeen
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Healthcare Quality and Management
- Keywords
- research design, improvement measures, randomized controlled trials, factorial design, block designs, before-after study, time series analysis, bias, generalizability, intervention development
- MeSH terms
- United Kingdom, Health Services Research, Humans, Organizational Innovation, Research Design, State Medicine, Reproducibility of Results, Program Evaluation, Cluster Analysis, Randomized Controlled Trials as Topic, Total Quality Management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
