Why modelling a complex intervention is an important precursor to trial design: lessons from studying an intervention to reduce falls-related injuries in older people
Open accessAbstract
Objectives: To develop a cost-effectiveness model of a complex intervention from pilot study data in order to inform the viability and design of a subsequent falls prevention trial. Methods: We used two models; the first estimated the probability of falling over a 12-month period based on a probability tree; the second used Markov simulation to assess the impact of the programme over time. Results: The first model indicated that our intervention would reduce the proportion falling by only 2.8% over a 12-month period. The major reason for this small effect was that less than a quarter of older people at risk of falling were assessed using our screening tool. Even if policy-makers were willing to spend £30,000 per quality-adjusted life-year gained, there is only a 40% chance that the intervention would be cost-effective. Sensitivity analyses showed that the only scenarios that produced a substantial increase in the effect of the intervention were those in which all older people are assessed. Conclusions: The model-building approach described in this paper is vital when designing complex trials and where a trial is not possible. Information from the modelling can be used to re-design the intervention. The effectiveness of our proposed intervention appears very small due to its inability to reach those at risk of falling. It is most likely not to be cost-effective. If inability to reach the target group is a weakness common to other similar interventions, this suggests an area for further research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two included studies reported on preventive interventions targeting older people and one on a programme to empower patients undergoing gynaecological surgery during the perioperative period.
“Two studies reported on preventive interventions targeting older people [24, 50] and one on a programme to empower patients undergoing gynaecological surgery during the perioperative period [43].”
From Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used (2016) · cites this paper as [50]
The remaining included studies did not specify the guidelines or frameworks adopted to develop the intervention of interest.
“The remaining studies did not specify the guidelines or frameworks adopted to develop the intervention of interest [48–50].”
From Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used (2016) · cites this paper as [50]
Factorial experiments, uncontrolled pilot studies and probability models tended to be used to test specific components and to identify an intervention's potential effect.
“For example, factorial experiments, uncontrolled pilot studies and probability models tended to be used to test specific components and to identify an intervention’s potential effect [10, 25, 29, 38, 44, 49, 50].”
From Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used (2016) · cites this paper as [50]
In two included studies, findings from optimisation processes discouraged researchers from moving to the full-scale RCT stage, as the interventions did not show the potential for effectiveness that was expected.
“In two studies, findings from optimisation processes discouraged researchers from moving to the full-scale RCT stage, as the interventions did not show the potential for effectiveness that was expected [35, 50].”
From Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used (2016) · cites this paper as [50]
Seven included studies were a preliminary investigation of the effect of the intervention or a combination of its components on proxy or final outcomes and intervention long-term impact.
“Seven studies were a preliminary investigation of the effect of the intervention or a combination of its components on proxy or final outcomes and intervention long-term impact [10, 24, 44–46, 49, 50].”
From Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used (2016) · cites this paper as [50]
Given current financial constraints upon health services research and the large number of trials that fail to show effectiveness, it is increasingly important to define strategies that support researchers in the development of more effective interventions.
“Given the current financial constraints upon health services research and the large number of trials that fail to show effectiveness, it has been suggested that it is increasingly important to define strategies that support researchers in the development of more effective interventions [3, 50, 51].”
From Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used (2016) · cites this paper as [50]
The MRC framework has been highly influential, with many additional publications on applying, extending, and revising the framework.
“It has been highly influential, with many additional publications on applying, extending, and revising the framework [92-94].”
From Web-Based Interventions for Behavior Change and Self-Management: Potential, Pitfalls, and Progress (2012) · cites this paper as [93]
Cites
1 retracted
Interventions for preventing falls in elderly people
Gillespie, L. D., Gillespie, W. J., Robertson, M. C., Lamb, S. E., Cumming, R. G., & Rowe, B. H. (2003) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000340 · Cited by 1,447 · see citing works
RetractedMethods for the Economic Evaluation of Health Care Programmes
Drummond, M. E., Sculpher, M. J., Torrance, G. W., O’Brien, B. J., & Stoddart, G. L. (2005) · doi:10.1093/oso/9780198529446.001.0001 · Cited by 13,648 · see citing works
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · see citing works
Markov Models in Medical Decision Making: A Practical Guide
Sonnenberg, F. A., & Beck, J. R. (1993) · Medical Decision Making · doi:10.1177/0272989x9301300409 · Cited by 2,419 · see citing works
(2001) · PubMed · Cited by 1,709 · see citing works
The green book : appraisal and evaluation in central government
農林水産奨励会農林水産政策情報センター, & 英国財務省 (2003) · 農林水産奨励会農林水産政策情報センター eBooks · Cited by 1,429 · see citing works
Stinnett, A. A., & Mullahy, J. (1998) · Medical Decision Making · doi:10.1177/0272989x98018002s09 · Cited by 1,103 · see citing works
Criteria for evaluating evidence on public health interventions
Rychetnik, L., Frommer, M., Hawe, P., & Shiell, A. (2002) · Journal of Epidemiology and Community Health · doi:10.1136/jech.56.2.119 · Cited by 1,042 · see citing works
Falls in Older People: Risk Factors and Strategies for Prevention
Lord, S. R., Sherrington, C., & Menz, H. B. (2000) · Cited by 367 · see citing works
Guidelines for the prevention of falls in people over 65
Feder, G. S. (2000) · BMJ · doi:10.1136/bmj.321.7267.1007 · Cited by 332 · see citing works
Self-citationSensitivity Analysis and the Expected Value of Perfect Information
Felli, J. C., & Hazen, G. B. (1998) · Medical Decision Making · doi:10.1177/0272989x9801800117 · Cited by 324 · see citing works
The Cost of Treating Osteoporotic Fractures in the United Kingdom Female Population
Dolan, P., & Torgerson, D. J. (1998) · Osteoporosis International · doi:10.1007/s001980050107 · Cited by 290 · see citing works
The tribulations of trials--intervention in communities.
Susser, M. W. (1995) · American Journal of Public Health · doi:10.2105/ajph.85.2.156 · Cited by 235 · see citing works
Bradley, F., Wiles, R., Kinmonth, A. L., Mant, D., & Gantley, M. (1999) · BMJ · doi:10.1136/bmj.318.7185.711 · Cited by 199 · see citing works
BAYESIAN VALUE-OF-INFORMATION ANALYSIS
Claxton, K., Neumann, P. J., Araki, S. S., & Weinstein, M. C. (2001) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462301104058 · Cited by 186 · see citing works
Discounting for health effects in cost–benefit and cost‐effectiveness analysis
Gravelle, H. S. E., & Smith, D. (2001) · Health Economics · doi:10.1002/hec.618 · Cited by 177 · see citing works
Lord, S. R., Menz, H. B., & Sherrington, C. (2004) · Springer eBooks · doi:10.1007/978-0-85729-402-9_12 · Cited by 137 · see citing works
Extending the CONSORT statement to cluster randomized trials: for discussion
Elbourne, D. R., & Campbell, M. (2001) · Statistics in Medicine · doi:10.1002/1097-0258(20010215)20:3 · Cited by 91 · see citing works
Economic modelling before clinical trials
Torgerson, D. J., & Byford, S. (2002) · BMJ · doi:10.1136/bmj.325.7355.98 · Cited by 17 · see citing works
Loomes, G. (2002) · UEA Digital Repository (University of East Anglia) · Cited by 16 · see citing works
A ''Ready Reckoner'' for Staff Costs in the NHS, Volume I, Estimated Costs
Netten, A., Knight, J., Dennett, J., Cooley, R. W., & Slight, A. (1998) · Kent Academic Repository (University of Kent) · Cited by 12 · see citing works
Carter, Y., Cryer, C., & Crown, J. (2002) · Kent Academic Repository (University of Kent) · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · see citing works
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2013) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2012.09.010 · Cited by 1,658 · see citing works
Murray, E., Treweek, S., Pope, C., MacFarlane, A., Ballini, L., Dowrick, C., Finch, T., Kennedy, A., Mair, F., O'Donnell, C., et al. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-63 · Cited by 1,478 · see citing works
Designing and evaluating complex interventions to improve health care
Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L. (2007) · BMJ · doi:10.1136/bmj.39108.379965.be · Cited by 1,331 · see citing works
Theorising interventions as events in systems
Hawe, P., Shiell, A., & Riley, T. (2009) · American Journal of Community Psychology · doi:10.1007/s10464-009-9229-9 · Cited by 1,143 · see citing works
Pilot and feasibility studies come of age!
Lancaster, G. A. (2015) · Pilot and Feasibility Studies · doi:10.1186/2055-5784-1-1 · Cited by 260 · see citing works
Web-Based Interventions for Behavior Change and Self-Management: Potential, Pitfalls, and Progress
Murray, E. (2012) · Medicine 2.0 · doi:10.2196/med20.1741 · Cited by 219 · see citing works
Levati, S., Campbell, P., Frost, R., Dougall, N., Wells, M., Donaldson, C., & Hagen, S. (2016) · Pilot and Feasibility Studies · doi:10.1186/s40814-016-0058-y · Cited by 101 · see citing works
Kolehmainen, N., & Francis, J. J. (2012) · Implementation Science · doi:10.1186/1748-5908-7-100 · Cited by 24 · see citing works
Treweek, S., & Sullivan, F. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-28 · Cited by 7 · see citing works
- Authors
- Eldridge, S., Spencer, A., Cryer, C., Parsons, S., Underwood, M., & Feder, G.
- Affiliations
- Queen Mary University of London (GB)
- University of Kent (GB)
- Venue
- Journal of Health Services Research & Policy
- Publisher
- SAGE Publications
- Published
- 2005-07-01
- Volume
- 10
- Issue
- 3
- Pages
- 133-142
- ISSN
- 1355-8196 (ISSN-L)
- 1758-1060
- OpenAlex
- W2073433200
- Cited by
- 58
- References
- 23
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 5 · 2022: 4 · 2021: 4 · 2020: 1 · 2019: 2 · 2018: 2 · 2016: 3 · 2015: 2 · 2014: 6 · 2013: 5 · 2012: 8
- FWCI
- 8.20
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Locations
- Journal of Health Services Research & Policy (publishedVersion)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Kent Academic Repository (University of Kent) (submittedVersion)
- Topics
- Health Sciences › Health Professions › Physical Therapy, Sports Therapy and Rehabilitation › Balance, Gait, and Falls Prevention
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- fall prevention, older adults, fall-related injuries, complex interventions, cost-effectiveness modelling, trial design, fall risk screening, probability trees, intervention feasibility, quality-adjusted life years
- MeSH terms
- Accidental Falls, Aged, Aged, 80 and over, Cost-Benefit Analysis, Decision Support Techniques, United Kingdom, Humans, Markov Chains, Pilot Projects, State Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
