Using theory-based evaluation to build evidence-based health and social care policy and practice
Abstract
Building an evidence base to underpin policy and practice in health and social care is one of the cornerstones of New Labour's modernization programme. However, the history of evaluation research in Britain and America suggests that this enterprise may be thwarted as much by politics as by methodological debates. The 'paradigm wars' that characterize the social sciences also feature in the field of evaluation research, and changes in the way 'health' is conceptualized in the 'new public health' have intensified debates about approaches to evaluation and what counts as evidence. Health Action Zones have been vaunted as both the 'flagships' of the government's health policy and as 'learning initiatives'. This paper uses the example of Health Action Zones to discuss the use of a theory-based approach to evaluation for the purpose of generating the policy learning needed to tackle health inequalities, drawing on experience in Plymouth. The limitations of the 'traditional' approach to evaluation are examined, along with recent guidance from the Medical Research Council about evaluating complex community initiatives. It is argued that the case for theory-based evaluation extends beyond criticism of the traditional approach to develop evaluation theory and practice from the perspective of critical realism. This approach to evaluation seeks explanation and understanding about how and why projects 'work' or not in specific contexts, rather than pragmatic solutions to de-contextualized problems. The paper concludes by reflecting on the politics of evaluation in the field of health and social care, calling into question the government's commitment to evidence-based policy and practice.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Other methods included case study approaches and iterative thematic analysis.
“Other methods included case study approaches [16, 33, 36, 64] and iterative thematic analysis [71] whereas others did not explicitly state their specific data analysis approach [14, 77].”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [36]
There were clear overlaps with other theory-driven evaluation approaches, in particular, realist approaches and logic models.
“However, there were clear overlaps with other theory-driven evaluation approaches, in particular, realist approaches [32, 36, 38, 56] and logic models [12, 26, 31, 52, 54, 61, 65, 66, 68, 69, 76, 82].”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [36]
Cites
(2014) · SAGE Publications Ltd eBooks · doi:10.4135/9781446294406.n148 · Cited by 7,272 · see citing works
Neufeld, A., & Ryan, A. G. (1988) · Canadian Journal of Program Evaluation · doi:10.3138/cjpe.3.006 · Cited by 5,906 · see citing works
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · see citing works
Campbell, D. T. (1969) · American Psychologist · doi:10.1037/h0027982 · Cited by 1,886 · see citing works
Why we need observational studies to evaluate the effectiveness of health care
Black, N. (1996) · BMJ · doi:10.1136/bmj.312.7040.1215 · Cited by 1,598 · see citing works
What works?Evidence-based policy and practice in public services
Davies, H. T. O., Nutley, S., & SMITH, P. C. (2000) · Policy Press eBooks · doi:10.1332/policypress/9781861341914.001.0001 · Cited by 912 · see citing works
National Service Framework for Mental Health
Thornicroft, G. (2000) · Psychiatric Bulletin · doi:10.1192/pb.24.6.203 · Cited by 639 · see citing works
Kubisch, J. P. C. A. A. C. (1998) · Cited by 461 · see citing works
Chen, H.-T., & Rossi, P. H. (1983) · Evaluation Review · doi:10.1177/0193841x8300700301 · Cited by 420 · see citing works
Evaluating information technology in health care: barriers and challenges
Heathfield, H., Pitty, D., & Hanka, R. (1998) · BMJ · doi:10.1136/bmj.316.7149.1959 · Cited by 226 · see citing works
Harré, R. (1985) · Oxford University Press eBooks · Cited by 222 · see citing works
Bhaskar, R. (2024) · Routledge eBooks · doi:10.4324/9781003572312 · Cited by 215 · see citing works
The Failure of Educational Evaluation.
Guba, E. G. (1969) · Cited by 71 · see citing works
Flagged first, then by citations
1 of the 14 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Using theory of change to design and evaluate public health interventions: a systematic review
Breuer, E., Lee, L., De Silva, M., & Lund, C. (2015) · Implementation Science · doi:10.1186/s13012-016-0422-6 · Cited by 425 · see citing works
Theories of Change and Realistic Evaluation: Peas in a Pod or Apples and Oranges?
Blamey, A., & Mackenzie, M. (2007) · Evaluation · doi:10.1177/1356389007082129 · Cited by 383 · see citing works
Berghs, M. J., Atkin, K. M., Graham, H. M., Hatton, C., & Thomas, C. (2016) · Public Health Research · doi:10.3310/phr04080 · Cited by 195 · see citing works
Clark, A. M., Lissel, S. L., & Davis, C. (2008) · Advances in Nursing Science · doi:10.1097/01.ans.0000341421.34457.2a · Cited by 140 · see citing works
Smith, K. E., Bambra, C. L., Joyce, K. E., Perkins, N., Hunter, D. J., & Blenkinsopp, E. (2009) · Journal of Public Health · doi:10.1093/pubmed/fdp002 · Cited by 111 · see citing works
Shaw, S. E. (2010) · Health An Interdisciplinary Journal for the Social Study of Health Illness and Medicine · doi:10.1177/1363459309353295 · Cited by 83 · see citing works
Football and social inclusion: Evaluating social policy
Tacon, R. (2007) · Managing Leisure · doi:10.1080/13606710601056422 · Cited by 71 · see citing works
‘What counts is what works’? New Labour and partnerships in public health
Perkins, N., Smith, K. E., Hunter, D. J., Bambra, C. L., & Joyce, K. E. (2010) · Policy & Politics · doi:10.1332/030557309x458425 · Cited by 71 · see citing works
Lee, J. C. L., Schram, A., Riley, E. J., Harris, P. J., Baum, F. E., Fisher, M., Freeman, T., & Friel, S. (2018) · International Journal of Health Policy and Management · doi:10.15171/ijhpm.2018.04 · Cited by 49 · see citing works
Dissemination and Implementation of Evidence-based Practice in the Social Services: A UK Survey
Morago, P. (2010) · Journal of Evidence-Based Social Work · doi:10.1080/15433714.2010.494973 · Cited by 44 · see citing works
Monitoring population‐level responses of marine mammals to human activities
Fleishman, E., da Costa, D. P., Harwood, J., Kraus, S. D., Moretti, D., New, L., Schick, R. S., Schwarz, L. K., Simmons, S. E., Thomas, L., et al. (2016) · Marine Mammal Science · doi:10.1111/mms.12310 · Cited by 33 · see citing works
Mitchell, F. (2014) · Health & Social Care in the Community · doi:10.1111/hsc.12137 · Cited by 22 · see citing works
Improving health and social care relationships for harm reduction
Allman, D., Myers, T., Schellenberg, J., Strıke, C., Cockerill, R. W., & Cavalieri, W. (2006) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2006.07.005 · Cited by 22 · see citing works
Dupin, C. M., Breton, É., Kivits, J., & Minary, L. (2015) · Santé Publique · doi:10.3917/spub.155.0653 · Cited by 20 · see citing works
Douglas, F., Gray, D., & van Teijlingen, E. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-49 · Cited by 22 · see citing works
Macdiarmid, J. I., Loe, J., Douglas, F., Ludbrook, A., Comerford, C., & McNeill, G. (2010) · Public Health Nutrition · doi:10.1017/s1368980010002168 · Cited by 17 · see citing works
Self‐directed support and disabled young people in transition (part 1)
Mitchell, F. (2012) · Journal of Integrated Care · doi:10.1108/14769011211202292 · Cited by 15 · see citing works
NHS health trainers: a review of emerging evaluation evidence
Attree, P., Clayton, S., Karunanithi, S., Nayak, S., Popay, J., & Read, D. (2011) · Critical Public Health · doi:10.1080/09581596.2010.549207 · Cited by 15 · see citing works
Rodriguez, L., George, J. R., & McDonald, B. (2017) · Kōtuitui New Zealand Journal of Social Sciences Online · doi:10.1080/1177083x.2017.1363059 · Cited by 14 · see citing works
The influence of professional values on the implementation of Aboriginal health policy
Lloyd, J. E., Wise, M., Weeramanthri, T. S., & Nugus, P. (2008) · Journal of Health Services Research & Policy · doi:10.1258/jhsrp.2008.008002 · Cited by 14 · see citing works
Self‐directed support and disabled young people in transition (part 2)
Mitchell, F. (2012) · Journal of Integrated Care · doi:10.1108/14769011211255258 · Cited by 11 · see citing works
Tucker, J. D., van Teijlingen, E., Philip, K., Shucksmith, J. S., & Penney, G. (2006) · Critical Public Health · doi:10.1080/09581590600986358 · Cited by 8 · see citing works
Evidence-based Practice in Social Work Education
Suvada, J., Olah, M., & Czarnecki, P. (2013) · Humanum. Międzynarodowe Studia Społeczno-Humanistyczne · Cited by 4 · see citing works
Diaries in health promotion research: the Mobile Information Bus evaluation
Wyness, L., Douglas, F., & van Teijlingen, E. (2004) · Health Education · doi:10.1108/09654280410560550 · Cited by 3 · see citing works
Roffey, S., & Barnes, J. (2014) · Well Being · doi:10.1002/9781118539415.wbwell047 · Cited by 3 · see citing works
Community change: the complex nature of interventions to promote positive connections
Roffey, S., & Barnes, J. (2014) · BIROn (Birkbeck, University of London) · Cited by 2 · see citing works
Implementing the ‘new scientific spirit’: a response to de Leuuwet al.
Crawshaw, P. (2008) · Critical Public Health · doi:10.1080/09581590802166601 · Cited by 2 · see citing works
Eaves, J., & Gnich, W. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-425 · Cited by 1 · see citing works
The National Healthy Schools Programme: A vehicle for school improvement? Case studies from Cheshire
Thurston, M. N. (2006) · ChesterRep (University of Chester) · Cited by 1 · see citing works
Morago, P. (2010) · Complutensian Scientific Journals (Complutense University of Madrid) · doi:10.5209/cuts.8320
Atangana, B. M. (2015) · University of Birmingham Institutional Research Archive (University of Birmingham)
SDEM: a supplier diversity initiative as a vehicle for supply-chain learning
Theodorakopoulos, N., Ram, M., & Boyal, H. (2006) · Aston Publications Explorer (Aston University)
Hussey, P. (2010) · Arrow@dit (Dublin Institute of Technology)
- Authors
- Bonner, L.
- Corresponding author
- Bonner, Lesley
- Venue
- Critical Public Health
- Publisher
- Informa UK Limited
- Published
- 2003-03
- Volume
- 13
- Issue
- 1
- Pages
- 77-92
- ISSN
- 0958-1596 (ISSN-L)
- 1469-3682
- OpenAlex
- W2030076762
- Cited by
- 33
- References
- 14
- Citations by year
- 2018: 1 · 2017: 1 · 2016: 2 · 2015: 3 · 2014: 3 · 2013: 2 · 2012: 2
- FWCI
- 2.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Critical Public Health (publishedVersion)
- University of Huddersfield Repository (University of Huddersfield) (acceptedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Public Health Policies and Education
- Keywords
- theory-based evaluation, evidence-based policy, health and social care policy, Health Action Zones, health disparities, policy learning, critical realism, paradigm wars, politics of evaluation, New Labour health policy, Plymouth
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
