Evidence, Understanding and Complexity: Evaluation in Non-Linear Systems
Open accessAbstract
This article discusses the challenges associated with the evaluation of complex policy initiatives, focusing on one element of the national evaluation of Health Action Zones in England: Building Capacity for Collaboration. It describes the multiple dimensions of complexity that characterize HAZs and considers the limitations of the Theory of Change approach in encompassing the diverse change processes that HAZs seek to promote. Drawing on social constructionist perspectives and from both complexity theory and new institutional theory, the authors propose that evaluators need not only to be able to work with change theories generated within such complex programmes, but also to apply `higher level' theory to an understanding of the way such programmes develop. They argue that the role of evaluators in `telling the story' is an important part of the evaluation challenge in such contexts.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Barnes et al. caution against interpreting context as a purely external factor, arguing that context is shaped by actors as much as it constrains their activities.
“Barnes et al., 128 for example, also caution against interpreting context as a purely external factor, arguing that context is shaped by actors as much as it constrains their activities.”
From ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study (2015) · cites this paper as [128]
Cites
Institutions, Institutional Change and Economic Performance
North, D. C. (1990) · doi:10.1017/cbo9780511808678 · Cited by 32,691 · see citing works
Evaluation in Complex Policy Systems
Sanderson, I. (2000) · Evaluation · doi:10.1177/13563890022209415 · Cited by 237 · see citing works
Strong theory, flexible methods: Evaluating complex community-based initiatives
Judge, K., & Bauld, L. (2001) · Critical Public Health · doi:10.1080/09581590010028237 · Cited by 135 · see citing works
The well-connected community: networking to the edge of chaos
Gilchrist, A. M. (2000) · Community Development Journal · doi:10.1093/cdj/35.3.264 · Cited by 114 · see citing works
‘With a Little Help From Our Friends’: Public-Private Partnerships as Institutions and Instruments
Peters, B. G. (1998) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-14408-2_2 · Cited by 112 · see citing works
Knowledge, Complexity, and Understanding
Cilliers, P. (2000) · Emergence · doi:10.1207/s15327000em0204_03 · Cited by 85 · see citing works
Building Collaborative Capacity through `Theories of Change'
Sullivan, H. T., Barnes, M., & Matka, E. (2002) · Evaluation · doi:10.1177/1358902002008002514 · Cited by 85 · see citing works
Self-citationFrom Programme Theory to Constructivism
Dahler‐Larsen, P. (2001) · Evaluation · doi:10.1177/13563890122209711 · Cited by 83 · see citing works
Change in public service management: New institutions and new managerial regimes
Lowndes, V. (1997) · Local Government Studies · doi:10.1080/03003939708433864 · Cited by 54 · see citing works
Concept Mapping in Policy Evaluation
Knox, C. G. (1995) · Evaluation · doi:10.1177/135638909500100105 · Cited by 21 · see citing works
Building capacity for collaboration in English Health Action Zones
Barnes, M., & Sullivan, H. T. (2002) · doi:10.46692/9781847425492.006 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions
Rogers, P. J. (2008) · Evaluation · doi:10.1177/1356389007084674 · Cited by 999 · see citing works
Lessons from Complex Interventions to Improve Health
Hawe, P. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031912-114421 · Cited by 607 · see citing works
Realist synthesis - an introduction
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004) · UCL Discovery (University College London) · Cited by 570 · see citing works
Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G. (2012) · Evaluation · doi:10.1177/1356389012442444 · Cited by 476 · see citing works
Theories of Change and Realistic Evaluation: Peas in a Pod or Apples and Oranges?
Blamey, A., & Mackenzie, M. (2007) · Evaluation · doi:10.1177/1356389007082129 · Cited by 383 · see citing works
How Do You Modernize a Health Service? A Realist Evaluation of Whole‐Scale Transformation in London
Greenhalgh, T., Humphrey, C., HUGHES, J., Macfarlane, F., Butler, C., & Pawson, R. (2009) · Milbank Quarterly · doi:10.1111/j.1468-0009.2009.00562.x · Cited by 364 · see citing works
ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study
Wilson, P., Mathie, E., Keenan, J., McNeilly, E., Goodman, C., Howe, A., Poland, F., Staniszewska, S., Kendall, S., Munday, D., et al. (2015) · Health Services and Delivery Research · doi:10.3310/hsdr03380 · Cited by 297 · see citing works
Evaluating complex social interventions in a complex world
Byrne, D. (2013) · Evaluation · doi:10.1177/1356389013495617 · Cited by 207 · see citing works
Realist RCTs of complex interventions – An oxymoron
Marchal, B., Westhorp, G., Wong, G., Belle, S. V., Greenhalgh, T., Kegels, G., & Pawson, R. (2013) · Social Science & Medicine · doi:10.1016/j.socscimed.2013.06.025 · Cited by 168 · see citing works
Using complexity-consistent theory for evaluating complex systems
Westhorp, G. (2012) · Evaluation · doi:10.1177/1356389012460963 · Cited by 163 · see citing works
A realist evaluation of the management of a well- performing regional hospital in Ghana
Marchal, B., Dedzo, M., & Kegels, G. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-24 · Cited by 156 · see citing works
de Souza, D. (2013) · Evaluation · doi:10.1177/1356389013485194 · Cited by 155 · see citing works
Measuring the economic and social impacts of local authority events
Wood, E. H. (2005) · International Journal of Public Sector Management · doi:10.1108/09513550510576143 · Cited by 131 · see citing works
Wilkinson, A., Kupers, R., & Mangalagiu, D. (2012) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2012.10.031 · Cited by 123 · see citing works
Applying complexity theory: A review to inform evaluation design
Walton, M. (2014) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2014.04.002 · Cited by 122 · see citing works
Systems thinking and complexity: considerations for health promoting schools
Rosas, S. R. (2015) · Health Promotion International · doi:10.1093/heapro/dav109 · Cited by 110 · see citing works
Evaluating the outcomes of co-production in local government
Brix, J., Krogstrup, H. K., & Mortensen, N. M. (2019) · Local Government Studies · doi:10.1080/03003930.2019.1702530 · Cited by 113 · see citing works
Promoting Social Change: The Experience of Health Action Zones in England
BAULD, L., JUDGE, K., BARNES, M., BENZEVAL, M., MACKENZIE, M., & SULLIVAN, H. (2005) · Journal of Social Policy · doi:10.1017/s0047279405008858 · Cited by 106 · see citing works
Self-citationSocial prescribing and classed inequality: A journey of upward health mobility?
Gibson, K., Pollard, T. M., & Moffatt, S. M. (2021) · Social Science & Medicine · doi:10.1016/j.socscimed.2021.114037 · Cited by 106 · see citing works
Unpacking complexity in public health interventions with the Actor–Network Theory
Bilodeau, A., & Potvin, L. (2016) · Health Promotion International · doi:10.1093/heapro/daw062 · Cited by 103 · see citing works
Complex, but not quite complex enough: The turn to the complexity sciences in evaluation scholarship
Mowles, C. (2014) · Evaluation · doi:10.1177/1356389014527885 · Cited by 100 · see citing works
Constructing Theories of Change: Methods and Sources
Mason, P., & Barnes, M. (2007) · Evaluation · doi:10.1177/1356389007075221 · Cited by 123 · see citing works
Self-citationBelle, S. V., Marchal, B., Dubourg, D., & Kegels, G. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-741 · Cited by 95 · see citing works
Gates, E. F. (2016) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2016.08.004 · Cited by 93 · see citing works
Burton, P., Goodlad, R., & Croft, J. (2006) · Evaluation · doi:10.1177/1356389006069136 · Cited by 85 · see citing works
Mackenzie, M., & Blamey, A. (2005) · Evaluation · doi:10.1177/1356389005055538 · Cited by 85 · see citing works
Participatory systems mapping for complex energy policy evaluation
Barbrook-Johnson, P., & Penn, A. S. (2021) · Evaluation · doi:10.1177/1356389020976153 · Cited by 79 · see citing works
Dickinson, H. (2006) · Health & Social Care in the Community · doi:10.1111/j.1365-2524.2006.00650.x · Cited by 78 · see citing works
Complexity Theory and Evaluation in Public Management
Haynes, P. (2008) · Public Management Review · doi:10.1080/14719030802002766 · Cited by 75 · see citing works
Evaluation use in the organizational context – changing focus to improve theory
Højlund, S. (2014) · Evaluation · doi:10.1177/1356389013516053 · Cited by 71 · see citing works
Essential drugs policy in three rural counties in China: What does a complexity lens add?
Xiao, Y., Zhao, K., Bishai, D. M., & Peters, D. H. (2012) · Social Science & Medicine · doi:10.1016/j.socscimed.2012.09.034 · Cited by 69 · see citing works
Rolfe, S. (2019) · Evaluation · doi:10.1177/1356389019835229 · Cited by 68 · see citing works
Spatial data infrastructures as complex adaptive systems
Grus, Ł., Crompvoets, J., & Bregt, A. K. (2010) · International Journal of Geographical Information Systems · doi:10.1080/13658810802687319 · Cited by 65 · see citing works
Evaluation and Negotiated Order
Callaghan, G. (2008) · Evaluation · doi:10.1177/1356389008095485 · Cited by 63 · see citing works
Implications of the Changing Conversation About Causality for Evaluators
Gates, E., & Dyson, L. (2016) · American Journal of Evaluation · doi:10.1177/1098214016644068 · Cited by 64 · see citing works
Abayneh, S., Lempp, H. K., & Hanlon, C. (2020) · Research Involvement and Engagement · doi:10.1186/s40900-019-0175-x · Cited by 59 · see citing works
Hamdani, Y., Jetha, A., & Norman, C. D. (2011) · Child Care Health and Development · doi:10.1111/j.1365-2214.2011.01313.x · Cited by 60 · see citing works
Evaluation and/as Social Relation
Abma, T., & Widdershoven, G. A. M. (2008) · Evaluation · doi:10.1177/1356389007087540 · Cited by 60 · see citing works
A Different Kind of Evidence? Looking for ‘What Works’ in Engaging Young Offenders
Prior, D., & Mason, P. (2010) · Youth Justice · doi:10.1177/1473225410381688 · Cited by 58 · see citing works
Multi-view SDI Assessment Framework
Grus, Ł., Crompvoets, J., & Bregt, A. K. (2007) · Socio-Environmental Systems Modeling · Cited by 58 · see citing works
Avaliação de iniciativas e programas intersetoriais em saúde: desafios e aprendizados
Magalhães, R., & Bodstein, R. (2009) · Ciência & Saúde Coletiva · doi:10.1590/s1413-81232009000300021 · Cited by 57 · see citing works
Orton, L., Halliday, E. C., Collins, M., Egan, M., Lewis, S., Ponsford, R., Powell, K., Salway, S., Townsend, A. F., Whitehead, M. M., et al. (2016) · Critical Public Health · doi:10.1080/09581596.2016.1250868 · Cited by 56 · see citing works
Health inequalities in England, Scotland and Wales: Stakeholders' accounts and policy compared
Harrington, B. E., Smith, K. E., Hunter, D. J., Marks, L., Blackman, T., McKee, L., Greene, A., Elliott, E., & Williams, G. H. (2008) · Public Health · doi:10.1016/j.puhe.2008.10.010 · Cited by 54 · see citing works
Addressing gender inequality in science: the multifaceted challenge of assessing impact
Schmidt, E. K., & Cacace, M. (2017) · Research Evaluation · doi:10.1093/reseval/rvx003 · Cited by 55 · see citing works
Expert views on applying complexity theory in evaluation: Opportunities and barriers
Walton, M. (2016) · Evaluation · doi:10.1177/1356389016667890 · Cited by 51 · see citing works
Hickey, G., McGilloway, S., O'Brien, M., Leckey, Y., Devlin, M., & Donnelly, M. (2018) · Children and Youth Services Review · doi:10.1016/j.childyouth.2018.06.008 · Cited by 47 · see citing works
Schmidt, E. K., & Graversen, E. K. (2019) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2019.101750 · Cited by 48 · see citing works
Rolfe, S. (2015) · Local Government Studies · doi:10.1080/03003930.2015.1081848 · Cited by 45 · see citing works
Herens, M. C., Wagemakers, A., Vaandrager, L., van Ophem, J., & Koelen, M. A. (2016) · Evaluation & the Health Professions · doi:10.1177/0163278716652940 · Cited by 45 · see citing works
Bonell, C., Warren, E., & Meléndez-Torres, G. (2022) · Evaluation · doi:10.1177/13563890221086309 · Cited by 40 · see citing works
Twinning “Practices of Change” With “Theory of Change”
Arensman, B., van Waegeningh, C., & van Wessel, M. (2017) · American Journal of Evaluation · doi:10.1177/1098214017727364 · Cited by 44 · see citing works
Masso, M. R., & McCarthy, G. (2009) · International Journal of Evidence-Based Healthcare · doi:10.1111/j.1744-1609.2009.00132.x · Cited by 42 · see citing works
Price, M., & Lau, F. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-309 · Cited by 40 · see citing works
Platt, S. D., McLean, J., McCollam, A., Blamey, A., Mackenzie, M., McDaid, D., Maxwell, M., Halliday, E. C., & Woodhouse, A. (2006) · Cited by 40 · see citing works
Rametsteiner, E., & Weiss, G. (2005) · Forest Policy and Economics · doi:10.1016/j.forpol.2005.07.005 · Cited by 40 · see citing works
A Review of the Status of Spatial Data Infrastructure Implementation in Africa
Makanga, P. T., & Smit, J. L. (2010) · South African Computer Journal · doi:10.18489/sacj.v45i0.36 · Cited by 39 · see citing works
A systems approach to policy evaluation
Caffrey, L., & Munro, E. (2017) · Evaluation · doi:10.1177/1356389017730727 · Cited by 39 · see citing works
Chinese martial arts and the Olympics: Analysing the policy of the International Wushu Federation
Han, Q.-S., Theeboom, M., & Zhu, D. H. (2020) · International Review for the Sociology of Sport · doi:10.1177/1012690220957177 · Cited by 39 · see citing works
Practical Dilemmas for Health Promotion Evaluation
Potvin, L., & McQueen, D. V. (2008) · Springer eBooks · doi:10.1007/978-0-387-79733-5_3 · Cited by 39 · see citing works
From rhetoric to reality: Barriers faced by Health For All initiatives
Berkeley, D., & Springett, J. (2006) · Social Science & Medicine · doi:10.1016/j.socscimed.2005.11.057 · Cited by 39 · see citing works
‘In the eye of the beholder’: perceptions of local impact in English Health Action Zones
Sullivan, H. T., Judge, K., & Sewel, K. (2004) · Social Science & Medicine · doi:10.1016/j.socscimed.2004.02.013 · Cited by 38 · see citing works
Self-citationDo we really care about unintended outcomes? An analysis of evaluation theory and practice
Jabeen, S. (2016) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2015.12.010 · Cited by 41 · see citing works
The Evaluation of Complex Infrastructure Projects
Gerrits, L., & Verweij, S. (2018) · Edward Elgar Publishing eBooks · doi:10.4337/9781783478422 · Cited by 60 · see citing works
Villeval, M., Bidault, E., Shoveller, J., Alias, F., Basson, J.-C., Frasse, C., Génolini, J., Pons, E., Verbiguié, D., Grosclaude, P., et al. (2016) · International Journal of Public Health · doi:10.1007/s00038-016-0809-9 · Cited by 37 · see citing works
Petch, A., Cook, A., & Miller, E. (2013) · Health & Social Care in the Community · doi:10.1111/hsc.12050 · Cited by 36 · see citing works
Embedding research in health systems: lessons from complexity theory
Caffrey, L., Wolfe, C. D. A., & McKevitt, C. (2016) · Health Research Policy and Systems · doi:10.1186/s12961-016-0128-x · Cited by 36 · see citing works
Transdisciplinary Knowledge Producing Teams: Toward a Complex Systems Perspective
Lotrecchiano, G. R., & Misra, S. (2018) · Informing Science The International Journal of an Emerging Transdiscipline · doi:10.28945/4086 · Cited by 36 · see citing works
Barbrook-Johnson, P., Castellani, B. C., Hills, D., Penn, A. S., & Gilbert, N. (2021) · Evaluation · doi:10.1177/1356389020976491 · Cited by 33 · see citing works
Developing an Evaluation for Crosscutting Policy Goals: The Accessibility Policy Assessment Tool
Bristow, G., Farrington, J. H., Shaw, J., & Richardson, T. (2008) · Environment and Planning A Economy and Space · doi:10.1068/a4092 · Cited by 33 · see citing works
How do policy evaluators understand complexity?
Barbrook-Johnson, P., Proctor, A., Giorgi, S., & Phillipson, J. (2020) · Evaluation · doi:10.1177/1356389020930053 · Cited by 33 · see citing works
Learning from seasoned evaluators: Implications of systems approaches for evaluation practice
Gates, E. (2017) · Evaluation · doi:10.1177/1356389017697613 · Cited by 31 · see citing works
Trois défis pour l'évaluation en promotion de la santé
Potvin, L., Bilodeau, A., & Gendron, S. (2008) · Promotion & Education · doi:10.1177/1025382308093991 · Cited by 30 · see citing works
Evaluating a community-based public health intervention using a complex systems approach
Matheson, A., Walton, M., Gray, R. J., Lindberg, K., Shanthakumar, M., Fyfe, C., Wehipeihana, N., & Borman, B. (2017) · Journal of Public Health · doi:10.1093/pubmed/fdx117 · Cited by 29 · see citing works
Jabeen, S. (2017) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2017.09.005 · Cited by 30 · see citing works
The Meaning of Effectiveness in Assessing Community Initiatives
Schweigert, F. J. (2006) · American Journal of Evaluation · doi:10.1177/1098214006294304 · Cited by 29 · see citing works
Verweij, S. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2646785 · Cited by 28 · see citing works
A programme theory for liaison mental health services in England
House, A. O., Guthrie, E., Walker, A., Hewsion, J., Trigwell, P. J., Brennan, C. A., Crawford, M., Czoski‐Murray, C., Fossey, M. J., Hulme, C. T., et al. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3539-2 · Cited by 26 · see citing works
Dunn, J. R., van der Meulen, E., O'Campo, P. J., & Muntaner, C. (2012) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2012.03.006 · Cited by 27 · see citing works
Evaluation of cash transfer programs in sub-Saharan Africa: A methodological review
Owusu‐Addo, E., Renzaho, A. M. N., & Smith, B. J. (2018) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2018.02.010 · Cited by 27 · see citing works
L’Outil diagnostique de l’action en partenariat: fondements, élaboration et validation
Bilodeau, A., Galarneau, M., Fournier, M., & Potvin, L. (2011) · Canadian Journal of Public Health · doi:10.1007/bf03404054 · Cited by 26 · see citing works
Mackenzie, M., Blamey, A., Halliday, E. C., Maxwell, M., McCollam, A., McDaid, D., MacLean, J., Woodhouse, A., & Platt, S. D. (2007) · BMC Public Health · doi:10.1186/1471-2458-7-146 · Cited by 25 · see citing works
Wicked Comparisons: Reflections on Cross-national Research about Health Inequalities in the UK
Blackman, T., Hunter, D. J., Marks, L., Harrington, B., Elliott, E., Williams, G. H., Greene, A., & Mckee, L. (2010) · Evaluation · doi:10.1177/1356389009350016 · Cited by 20 · see citing works
Using the Objective Hermeneutics Method in Policy Evaluation
Mann, S., & Schweiger, J. (2009) · Evaluation · doi:10.1177/1356389009345448 · Cited by 23 · see citing works
The Social Relations of Evaluation
Abma, T. (2006) · SAGE Publications Ltd eBooks · doi:10.4135/9781848608078.n8 · Cited by 23 · see citing works
The complexity of the discussion on effectiveness and evidence in health promotion practices
Bodstein, R. (2007) · Promotion & Education · doi:10.1177/10253823070140010401x · Cited by 21 · see citing works
Orton, L., Ponsford, R., Egan, M., Halliday, E. C., Whitehead, M. M., & Popay, J. (2019) · Anthropology and Medicine · doi:10.1080/13648470.2018.1508639 · Cited by 21 · see citing works
Guenther, J., Benveniste, T., Redman-MacLaren, M. L., Mander, D. J., McCalman, J., O’Bryan, M., Osborne, S., & Stewart, R. (2020) · Evaluation Journal of Australasia · doi:10.1177/1035719x20905056 · Cited by 21 · see citing works
Focusing on Outcomes: Their Role in Partnership Policy and Practice
Petch, A., Cook, A., & Miller, E. (2005) · Journal of Integrated Care · doi:10.1108/14769018200500043 · Cited by 19 · see citing works
Taking stock of complexity in evaluation: A discussion of three recent publications
Gerrits, L., & Verweij, S. (2015) · Evaluation · doi:10.1177/1356389015605204 · Cited by 19 · see citing works
"Healthy Eating - Healthy Action": evaluating New Zealand's obesity prevention strategy
McLean, R. M., Hoek, J. A., Buckley, S., Croxson, B., Cumming, J., Ehau, T. H., Tanuvasa, A. F., Johnston, M., Mann, J. I., & Schofield, G. (2009) · BMC Public Health · doi:10.1186/1471-2458-9-452 · Cited by 18 · see citing works
Towards Modelling Data Infrastructures in the Asset Management Domain
Brous, P., Herder, P., & Janssen, M. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.09.215 · Cited by 18 · see citing works
A Complex Adaptive System of Systems Approach to Human–Automation Interaction in Smart Grid
Fereidunian, A., Lesani, H., Zamani, M. A., Kolarijani, M. A. S., Hassanpour, N., & Mansouri, S. S. (2015) · doi:10.1002/9781119036821.ch12 · Cited by 18 · see citing works
Gadsby, E. W., Hotham, S., Eida, T., Lawrence, C., & Merritt, R. (2020) · BMC Public Health · doi:10.1186/s12889-020-09694-2 · Cited by 17 · see citing works
Gibson, K., Moffatt, S. M., & Pollard, T. M. (2022) · Sociology of Health & Illness · doi:10.1111/1467-9566.13482 · Cited by 16 · see citing works
Moffatt, S. M., Wildman, J., Pollard, T. M., Gibson, K., Wildman, J. M., O’Brien, N., Griffith, B., Morris, S. L., Moloney, E., Jeffries, J. M., et al. (2023) · Public Health Research · doi:10.3310/aqxc8219 · Cited by 17 · see citing works
The Limits of Realist Evaluation
Davis, P. K. (2005) · Evaluation · doi:10.1177/1356389005058476 · Cited by 17 · see citing works
Avaliação da Política Nacional de Promoção da Saúde: perspectivas e desafios
Magalhães, R. (2016) · Ciência & Saúde Coletiva · doi:10.1590/1413-81232015216.07422016 · Cited by 16 · see citing works
Brous, P., Janssen, M., & Herder, P. (2019) · Complexity · doi:10.1155/2019/5415828 · Cited by 16 · see citing works
L’efficacité enseignante, quelle modélisation pour servir cette ambition ?
Clanet, J. (2012) · Questions vives recherches en éducation · doi:10.4000/questionsvives.1121 · Cited by 15 · see citing works
System change evaluation: Insights from The Rippel Foundation and its ReThink Health initiative
Gates, E., & Fils‐Aime, F. (2021) · New Directions for Evaluation · doi:10.1002/ev.20462 · Cited by 15 · see citing works
- Authors
- Barnes, M., Matka, E., & Sullivan, H.
- Affiliations
- University of Birmingham (GB)
- University of the West of England (GB)
- Corresponding author
- Barnes, Marian
- Matka, Elizabeth
- Sullivan, Helen
- ORCID
- Sullivan, Helen
- Venue
- Evaluation
- Publisher
- SAGE Publications
- Published
- 2003-07
- Volume
- 9
- Issue
- 3
- Pages
- 265-284
- ISSN
- 1356-3890 (ISSN-L)
- 1461-7153
- OpenAlex
- W2056268972
- Cited by
- 208
- References
- 11
- Citations by year
- 2026: 5 · 2025: 1 · 2024: 4 · 2023: 9 · 2022: 5 · 2021: 8 · 2020: 13 · 2019: 9 · 2018: 13 · 2017: 11 · 2016: 16 · 2015: 14 · 2014: 7 · 2013: 15 · 2012: 11
- FWCI
- 9.56
- Type
- journal article
- Language
- en
- Open access
- green
- eprints.uwe.ac.uk
- Locations
- Evaluation (publishedVersion)
- UWE Research Repository (UWE Bristol) (submittedVersion, cc-by)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- University of Brighton Repository (University of Brighton) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Social Sciences › Public Administration › Public Policy and Administration Research
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- Health Action Zones, theory of change, complexity theory, new institutionalism, social constructionism, evaluator role, program development, evidence-based policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
