The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Research designs for studies evaluating the effectiveness of change and improvement strategies
Open accessAbstract
The methods of evaluating change and improvement strategies are not well described. The design and conduct of a range of experimental and non-experimental quantitative designs are considered. Such study designs should usually be used in a context where they build on appropriate theoretical, qualitative and modelling work, particularly in the development of appropriate interventions. A range of experimental designs are discussed including single and multiple arm randomised controlled trials and the use of more complex factorial and block designs. The impact of randomisation at both group and individual levels and three non-experimental designs (uncontrolled before and after, controlled before and after, and time series analysis) are also considered. The design chosen will reflect both the needs (and resources) in any particular circumstances and also the purpose of the evaluation. The general principle underlying the choice of evaluative design is, however, simple-those conducting such evaluations should use the most robust design possible to minimise bias and maximise generalisability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The interventions may have been poorly developed, lacking appropriate theoretical basis and subsequent components to impact on the measured outcome domains [82,83].”
From Interventions targeting social isolation in older people: a systematic review (2011) · cites this paper as [82]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1963) · Medical Entomology and Zoology · Cited by 15,085
(1979) · Medical Entomology and Zoology · Cited by 11,252
Quasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,091
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474
Clinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,918
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033
Effectiveness and Efficiency: Random Reflections on Health Services
L'Etang, H. (1973) · Public Health · doi:10.1016/s0033-3506(73)80082-4 · Cited by 1,905
Explanatory and Pragmatic Attitudes in Therapeutical Trials
Schwartz, D., & Lellouch, J. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.01.012 · Cited by 1,253
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236
Time-Series Analysis: Regression Techniques.
Pampel, F. C., & Ostrom, C. W. (1980) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2065616 · Cited by 499
Design and Analysis of Cluster Randomization Trials in Health Research.
Flynn, T. N. (2001) · International Journal of Epidemiology · doi:10.1093/ije/30.2.407-a · Cited by 479
Eccles, M. (2002) · BMJ · doi:10.1136/bmj.325.7370.941 · Cited by 354
Self-citationAnalysis of cluster randomized trials in primary care: a practical approach
Campbell, M. K., Mollison, J., Steen, N., Grimshaw, J. M., & Eccles, M. (2000) · Family Practice · doi:10.1093/fampra/17.2.192 · Cited by 327
Self-citationThe Estimation of Intraclass Correlation in the Analysis of Family Data
Donner, A. P., & Koval, J. J. (1980) · Biometrics · doi:10.2307/2530491 · Cited by 305
Eccles, M., Steen, N., Grimshaw, J., Thomas, L., McNamee, P., Soutter, J., Wilsdon, J., Matowe, L., Needham, G., Gilbert, F., et al. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04564-5 · Cited by 222
Self-citationSome Aspects of the Design and Analysis of Cluster Randomization Trials
Donner, A. P. (1998) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/1467-9876.00100 · Cited by 178
Matowe, L. K., Ramsay, C. R., Grimshaw, J., Gilbert, F. J., MacLeod, M. J., & Needham, G. (2002) · Clinical Radiology · doi:10.1053/crad.2001.0894 · Cited by 78
The miscarriage clinic: an audit of the first year
Turner, M. J., Flannelly, G., Wingfield, M. B., Rasmussen, M., Ryan, R., Cullen, S., Maguire, R., & STRONGE, J. M. (1991) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.1991.tb13398.x · Cited by 60
Economic and Medical Evaluation of Health Care Technologies
Culyer, A. J., & Horisberger, B. (1983) · Springer eBooks · doi:10.1007/978-3-642-69439-4 · Cited by 59
Lobo, C. M., Frijling, B. D., Hulscher, M., Bernsen, R. M. D., Braspenning, J. C. C., Grol, R. P. T. M., Prins, A., & van der Wouden, J. C. (2002) · Preventive Medicine · doi:10.1006/pmed.2002.1095 · Cited by 43
Mollison, J., Simpson, J. A., Campbell, M., & Grimshaw, J. (2000) · PubMed · Cited by 42
Randomization by Group in Studying the Effect of Drug Information in Primary Care
Diwan, V. K., Eriksson, B., Sterky, G., & Tomson, G. B. (1992) · International Journal of Epidemiology · doi:10.1093/ije/21.1.124 · Cited by 42
Computerized concurrent review of prenatal care.
Morgan, M., Studney, D. R., Go, B., & Winickoff, R. N. (1978) · PubMed · Cited by 41
Recruitment strategies in a cluster randomized trial – cost implications
Flynn, T. N., Whitley, E., & Peters, T. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1025 · Cited by 40
Flagged first, then by citations
Research works printed but not linked: 6
- Murray DM. The design and analysis of group randomised trials. Oxford: Oxford University Press, 1998.
- Cochran WG, Cox GM. Experimental design. New York: Wiley, 1957.
- Eccles M, McColl E, Steen N, et al. A cluster randomised controlled trial of computerised evidence based guidelines for angina and asthma in primary care. BMJ 2002;325:941-7.
- Matowe L, Ramsay C, Grimshaw JM, et al. Influence of the Royal College of Radiologists' guidelines on referrals from general practice: a time series analysis. Clin Radiol 2002;57:575-8.
- Grilli R, Ramsay CR, Minozi S. Mass media interventions: effects on health services utilisation In: Cochrane Collaboration. The Cochrane Library. Oxford: Update Software, 2002.
- Grilli R, Freemantle N, Minozzi S, et al. Impact of mass media on health services utilisation. In: Cochrane Collaboration. The Cochrane Library. Issue 3. Oxford: Update Software, 1998.
Cited by
Sorted by citations
Developing and Evaluating Complex Interventions: New Guidance
Craig, P., Dieppe, P. A., MacIntyre, S., Michie, S., Nazareth, I., & Petticrew, M. P. (2006) · UCL Discovery (University College London) · Cited by 1,878
Qualitative and Mixed Methods Provide Unique Contributions to Outcomes Research
Curry, L. A., Nembhard, I. M., & Bradley, E. H. (2009) · Circulation · doi:10.1161/circulationaha.107.742775 · Cited by 1,175
Local opinion leaders: effects on professional practice and health care outcomes
Flodgren, G., Parmelli, E., Doumit, G., Gattellari, M., O'Brien, M. A., Grimshaw, J., & Eccles, M. P. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000125.pub4 · Cited by 1,070
Self-citationA systematic review of primary prevention strategies for sexual violence perpetration
DeGue, S. A., Valle, L. A., Holt, M. K., Massetti, G. M., Matjasko, J. L., & Tharp, A. T. (2014) · Aggression and Violent Behavior · doi:10.1016/j.avb.2014.05.004 · Cited by 781
Interventions targeting social isolation in older people: a systematic review
Dickens, A. P., Richards, S. H., Greaves, C. J., & Campbell, J. L. (2011) · BMC Public Health · doi:10.1186/1471-2458-11-647 · Cited by 767
Metadata
- Authors
- Eccles, M.
- Affiliations
- Newcastle University (GB)
- Corresponding author
- Eccles, M
- ORCID
- Eccles, M
- Venue
- Quality and Safety in Health Care
- Publisher
- BMJ
- Published
- 2003-02-01
- Volume
- 12
- Issue
- 1
- Pages
- 47-52
- ISSN
- 1475-3898 (ISSN-L)
- 1475-3901
- PMC
- PMC1743658
- OpenAlex
- W2046947214
- Cited by
- 586
- References
- 24
- Citations by year
- 2026: 8 · 2025: 11 · 2024: 16 · 2023: 21 · 2022: 16 · 2021: 24 · 2020: 34 · 2019: 44 · 2018: 31 · 2017: 40 · 2016: 35 · 2015: 44 · 2014: 32 · 2013: 24 · 2012: 23
- FWCI
- 35.41
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- other-oa
- Locations
- BMJ Quality & Safety (publishedVersion, other-oa)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Newcastle University
- University of Aberdeen
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Healthcare Quality and Management
- Keywords
- research design, improvement measures, randomized controlled trials, factorial design, block designs, before-after study, time series analysis, bias, generalizability, intervention development
- MeSH terms
- United Kingdom, Health Services Research, Humans, Organizational Innovation, Research Design, State Medicine, Reproducibility of Results, Program Evaluation, Cluster Analysis, Randomized Controlled Trials as Topic, Total Quality Management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
