Cited by 208Evaluation and Performance AssessmentPublic Policy and Administration ResearchHealth Policy Implementation Science

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Evidence, Understanding and Complexity: Evaluation in Non-Linear Systems

Open access

Barnes, Marian, Matka, Elizabeth, Sullivan, Helen·2003·Evaluation

DOI 10.1177/13563890030093003

Open this paper in CoChat — free account

Abstract

This article discusses the challenges associated with the evaluation of complex policy initiatives, focusing on one element of the national evaluation of Health Action Zones in England: Building Capacity for Collaboration. It describes the multiple dimensions of complexity that characterize HAZs and considers the limitations of the Theory of Change approach in encompassing the diverse change processes that HAZs seek to promote. Drawing on social constructionist perspectives and from both complexity theory and new institutional theory, the authors propose that evaluators need not only to be able to work with change theories generated within such complex programmes, but also to apply `higher level' theory to an understanding of the way such programmes develop. They argue that the role of evaluators in `telling the story' is an important part of the evaluation challenge in such contexts.

Integrity signals

Registry

Crossref · 10.1177/13563890030093003

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Institutions, Institutional Change and Economic Performance
North, D. C. (1990) · doi:10.1017/cbo9780511808678 · Cited by 32,691
Evaluation in Complex Policy Systems
Sanderson, I. (2000) · Evaluation · doi:10.1177/13563890022209415 · Cited by 237
Strong theory, flexible methods: Evaluating complex community-based initiatives
Judge, K., & Bauld, L. (2001) · Critical Public Health · doi:10.1080/09581590010028237 · Cited by 135
The well-connected community: networking to the edge of chaos
Gilchrist, A. M. (2000) · Community Development Journal · doi:10.1093/cdj/35.3.264 · Cited by 114
‘With a Little Help From Our Friends’: Public-Private Partnerships as Institutions and Instruments
Peters, B. G. (1998) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-14408-2_2 · Cited by 112
Knowledge, Complexity, and Understanding
Cilliers, P. (2000) · Emergence · doi:10.1207/s15327000em0204_03 · Cited by 85
Building Collaborative Capacity through `Theories of Change'
Sullivan, H. T., Barnes, M., & Matka, E. (2002) · Evaluation · doi:10.1177/1358902002008002514 · Cited by 85
Self-citation
From Programme Theory to Constructivism
Dahler‐Larsen, P. (2001) · Evaluation · doi:10.1177/13563890122209711 · Cited by 83
Change in public service management: New institutions and new managerial regimes
Lowndes, V. (1997) · Local Government Studies · doi:10.1080/03003939708433864 · Cited by 54
Concept Mapping in Policy Evaluation
Knox, C. G. (1995) · Evaluation · doi:10.1177/135638909500100105 · Cited by 21

Show all 11 references

Research works printed but not linked: 9

Bauld, L and K. Judge (2002) `Introduction: The Development of Health Action Zones', in L. Bauld and K. Judge (eds) Learning from Health Action Zones. Chichester: Aeneas Press .
Berger, P. L. and T. Luckmann (1973) The Social Construction of Reality. Harmondsworth: Penguin .
Glendinning, G., M. Powell and K. Rummery, eds (2002) Partnerships under New Labour: a Third Way Approach to Delivering Welfare. Bristol: Policy Press .
Immergut, E. (1992) Health Politics: Interests and Institutions in Western Europe. Cambridge: Cambridge University Press .
Judge, K., M. Barnes, L. Bauld, M. Benzeval, A. Killoran, R. Robinson, R. Wigglesworth and H. Zeilig (1999) Health Action Zones: Learning to Make a Difference, Findings from a Preliminary Review of Health Action Zones and Proposals for a National Evaluation, PSSRU Discussion Paper 1815. Canterbury: University of Kent .
Kauffman, S. (1995) At Home in the Universe. London: Penguin .
March, J. and J. Olsen (1989) Rediscovering Institutions. New York: Free Press .
Travers, T. (1993) `Professionals and Local Government Reform - Not So Much Villains as Saviours', in G. Stoker (ed.) The Future of Professionalism in Local Government, pp. 37-48. Luton: LGMB .
Williamson, O. E. (1985) The Economic Institutions of Capitalism. New York: Free Press .

Sorted by citations

Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions
Rogers, P. J. (2008) · Evaluation · doi:10.1177/1356389007084674 · Cited by 999
Lessons from Complex Interventions to Improve Health
Hawe, P. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031912-114421 · Cited by 607
Realist synthesis - an introduction
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004) · UCL Discovery (University College London) · Cited by 570
Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research
Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G. (2012) · Evaluation · doi:10.1177/1356389012442444 · Cited by 476
Theories of Change and Realistic Evaluation: Peas in a Pod or Apples and Oranges?
Blamey, A., & Mackenzie, M. (2007) · Evaluation · doi:10.1177/1356389007082129 · Cited by 383
How Do You Modernize a Health Service? A Realist Evaluation of Whole‐Scale Transformation in London
Greenhalgh, T., Humphrey, C., HUGHES, J., Macfarlane, F., Butler, C., & Pawson, R. (2009) · Milbank Quarterly · doi:10.1111/j.1468-0009.2009.00562.x · Cited by 364
ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study
Wilson, P., Mathie, E., Keenan, J., McNeilly, E., Goodman, C., Howe, A., Poland, F., Staniszewska, S., Kendall, S., Munday, D., et al. (2015) · Health Services and Delivery Research · doi:10.3310/hsdr03380 · Cited by 297
Evaluating complex social interventions in a complex world
Byrne, D. (2013) · Evaluation · doi:10.1177/1356389013495617 · Cited by 207
Realist RCTs of complex interventions – An oxymoron
Marchal, B., Westhorp, G., Wong, G., Belle, S. V., Greenhalgh, T., Kegels, G., & Pawson, R. (2013) · Social Science & Medicine · doi:10.1016/j.socscimed.2013.06.025 · Cited by 168
Using complexity-consistent theory for evaluating complex systems
Westhorp, G. (2012) · Evaluation · doi:10.1177/1356389012460963 · Cited by 163
A realist evaluation of the management of a well- performing regional hospital in Ghana
Marchal, B., Dedzo, M., & Kegels, G. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-24 · Cited by 156
Elaborating the Context-Mechanism-Outcome configuration (CMOc) in realist evaluation: A critical realist perspective
de Souza, D. (2013) · Evaluation · doi:10.1177/1356389013485194 · Cited by 155
Measuring the economic and social impacts of local authority events
Wood, E. H. (2005) · International Journal of Public Sector Management · doi:10.1108/09513550510576143 · Cited by 131
How plausibility-based scenario practices are grappling with complexity to appreciate and address 21st century challenges
Wilkinson, A., Kupers, R., & Mangalagiu, D. (2012) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2012.10.031 · Cited by 142
Applying complexity theory: A review to inform evaluation design
Walton, M. (2014) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2014.04.002 · Cited by 122
Systems thinking and complexity: considerations for health promoting schools
Rosas, S. R. (2015) · Health Promotion International · doi:10.1093/heapro/dav109 · Cited by 110
Evaluating the outcomes of co-production in local government
Brix, J., Krogstrup, H. K., & Mortensen, N. M. (2019) · Local Government Studies · doi:10.1080/03003930.2019.1702530 · Cited by 113
Promoting Social Change: The Experience of Health Action Zones in England
BAULD, L., JUDGE, K., BARNES, M., BENZEVAL, M., MACKENZIE, M., & SULLIVAN, H. (2005) · Journal of Social Policy · doi:10.1017/s0047279405008858 · Cited by 106
Self-citation
Social prescribing and classed inequality: A journey of upward health mobility?
Gibson, K., Pollard, T. M., & Moffatt, S. (2021) · Social Science & Medicine · doi:10.1016/j.socscimed.2021.114037 · Cited by 117
Unpacking complexity in public health interventions with the Actor–Network Theory
Bilodeau, A., & Potvin, L. (2016) · Health Promotion International · doi:10.1093/heapro/daw062 · Cited by 103
Complex, but not quite complex enough: The turn to the complexity sciences in evaluation scholarship
Mowles, C. (2014) · Evaluation · doi:10.1177/1356389014527885 · Cited by 100
Constructing Theories of Change: Methods and Sources
Mason, P., & Barnes, M. (2007) · Evaluation · doi:10.1177/1356389007075221 · Cited by 123
Self-citation
How to develop a theory-driven evaluation design? Lessons learned from an adolescent sexual and reproductive health programme in West Africa
Belle, S. V., Marchal, B., Dubourg, D., & Kegels, G. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-741 · Cited by 95
Making sense of the emerging conversation in evaluation about systems thinking and complexity science
Gates, E. F. (2016) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2016.08.004 · Cited by 106
How Would We Know What Works?
Burton, P., Goodlad, R., & Croft, J. (2006) · Evaluation · doi:10.1177/1356389006069136 · Cited by 89
The Practice and the Theory
Mackenzie, M., & Blamey, A. (2005) · Evaluation · doi:10.1177/1356389005055538 · Cited by 85
Participatory systems mapping for complex energy policy evaluation
Barbrook-Johnson, P., & Penn, A. S. (2021) · Evaluation · doi:10.1177/1356389020976153 · Cited by 79
The evaluation of health and social care partnerships: an analysis of approaches and synthesis for the future
Dickinson, H. (2006) · Health & Social Care in the Community · doi:10.1111/j.1365-2524.2006.00650.x · Cited by 78
Complexity Theory and Evaluation in Public Management
Haynes, P. (2008) · Public Management Review · doi:10.1080/14719030802002766 · Cited by 75
Evaluation use in the organizational context – changing focus to improve theory
Højlund, S. (2014) · Evaluation · doi:10.1177/1356389013516053 · Cited by 71
Essential drugs policy in three rural counties in China: What does a complexity lens add?
Xiao, Y., Zhao, K., Bishai, D. M., & Peters, D. H. (2012) · Social Science & Medicine · doi:10.1016/j.socscimed.2012.09.034 · Cited by 72
Combining Theories of Change and Realist Evaluation in practice: Lessons from a research on evaluation study
Rolfe, S. (2019) · Evaluation · doi:10.1177/1356389019835229 · Cited by 68
Spatial data infrastructures as complex adaptive systems
Grus, Ł., Crompvoets, J., & Bregt, A. K. (2010) · International Journal of Geographical Information Systems · doi:10.1080/13658810802687319 · Cited by 65
Evaluation and Negotiated Order
Callaghan, G. (2008) · Evaluation · doi:10.1177/1356389008095485 · Cited by 63
Implications of the Changing Conversation About Causality for Evaluators
Gates, E., & Dyson, L. (2016) · American Journal of Evaluation · doi:10.1177/1098214016644068 · Cited by 64
Participatory action research to pilot a model of mental health service user involvement in an Ethiopian rural primary healthcare setting: study protocol
Abayneh, S., Lempp, H. K., & Hanlon, C. (2020) · Research Involvement and Engagement · doi:10.1186/s40900-019-0175-x · Cited by 62
Systems thinking perspectives applied to healthcare transition for youth with disabilities: a paradigm shift for practice, policy and research
Hamdani, Y., Jetha, A., & Norman, C. D. (2011) · Child Care Health and Development · doi:10.1111/j.1365-2214.2011.01313.x · Cited by 64
Evaluation and/as Social Relation
Abma, T., & Widdershoven, G. A. M. (2008) · Evaluation · doi:10.1177/1356389007087540 · Cited by 60
A Different Kind of Evidence? Looking for ‘What Works’ in Engaging Young Offenders
Prior, D., & Mason, P. (2010) · Youth Justice · doi:10.1177/1473225410381688 · Cited by 58
Multi-view SDI Assessment Framework
Grus, Ł., Crompvoets, J., & Bregt, A. K. (2007) · Socio-Environmental Systems Modeling · Cited by 58
Avaliação de iniciativas e programas intersetoriais em saúde: desafios e aprendizados
Magalhães, R., & Bodstein, R. (2009) · Ciência & Saúde Coletiva · doi:10.1590/s1413-81232009000300021 · Cited by 57
Putting context centre stage: evidence from a systems evaluation of an area based empowerment initiative in England
Orton, L., Halliday, E. C., Collins, M., Egan, M., Lewis, S., Ponsford, R., Powell, K., Salway, S., Townsend, A. F., Whitehead, M. M., et al. (2016) · Critical Public Health · doi:10.1080/09581596.2016.1250868 · Cited by 56
Health inequalities in England, Scotland and Wales: Stakeholders' accounts and policy compared
Harrington, B. E., Smith, K. E., Hunter, D. J., Marks, L., Blackman, T., McKee, L., Greene, A., Elliott, E., & Williams, G. H. (2008) · Public Health · doi:10.1016/j.puhe.2008.10.010 · Cited by 54
Addressing gender inequality in science: the multifaceted challenge of assessing impact
Schmidt, E. K., & Cacace, M. (2017) · Research Evaluation · doi:10.1093/reseval/rvx003 · Cited by 55
Expert views on applying complexity theory in evaluation: Opportunities and barriers
Walton, M. (2016) · Evaluation · doi:10.1177/1356389016667890 · Cited by 51
Strengthening stakeholder buy-in and engagement for successful exploration and installation: A case study of the development of an area-wide, evidence-based prevention and early intervention strategy
Hickey, G., McGilloway, S., O'Brien, M., Leckey, Y., Devlin, M., & Donnelly, M. (2018) · Children and Youth Services Review · doi:10.1016/j.childyouth.2018.06.008 · Cited by 47
Developing a conceptual evaluation framework for gender equality interventions in research and innovation
Schmidt, E. K., & Graversen, E. K. (2019) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2019.101750 · Cited by 55
Divergence in Community Participation Policy: Analysing Localism and Community Empowerment Using a Theory of Change Approach
Rolfe, S. (2015) · Local Government Studies · doi:10.1080/03003930.2015.1081848 · Cited by 45
Contexts, Mechanisms, and Outcomes That Matter in Dutch Community-Based Physical Activity Programs Targeting Socially Vulnerable Groups
Herens, M. C., Wagemakers, A., Vaandrager, L., van Ophem, J., & Koelen, M. A. (2016) · Evaluation & the Health Professions · doi:10.1177/0163278716652940 · Cited by 45
Methodological reflections on using qualitative research to explore the causal mechanisms of complex health interventions
Bonell, C., Warren, E., & Meléndez-Torres, G. (2022) · Evaluation · doi:10.1177/13563890221086309 · Cited by 40
Twinning “Practices of Change” With “Theory of Change”
Arensman, B., van Waegeningh, C., & van Wessel, M. (2017) · American Journal of Evaluation · doi:10.1177/1098214017727364 · Cited by 44
Literature review to identify factors that support implementation of evidence-based practice in residential aged care
Masso, M. R., & McCarthy, G. (2009) · International Journal of Evidence-Based Healthcare · doi:10.1111/j.1744-1609.2009.00132.x · Cited by 42
Provider connectedness and communication patterns: extending continuity of care in the context of the circle of care
Price, M., & Lau, F. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-309 · Cited by 40
Evaluation of the First Phase of Choose Life: The National Strategy and Action Plan to Prevent Suicide in Scotland
Platt, S. D., McLean, J., McCollam, A., Blamey, A., Mackenzie, M., McDaid, D., Maxwell, M., Halliday, E. C., & Woodhouse, A. (2006) · Cited by 40
Assessing policies from a systems perspecitve — Experiences with applied innovation systems analysis and implications for policy evaluation
Rametsteiner, E., & Weiss, G. (2005) · Forest Policy and Economics · doi:10.1016/j.forpol.2005.07.005 · Cited by 40
A Review of the Status of Spatial Data Infrastructure Implementation in Africa
Makanga, P. T., & Smit, J. L. (2010) · South African Computer Journal · doi:10.18489/sacj.v45i0.36 · Cited by 39
A systems approach to policy evaluation
Caffrey, L., & Munro, E. (2017) · Evaluation · doi:10.1177/1356389017730727 · Cited by 39
Chinese martial arts and the Olympics: Analysing the policy of the International Wushu Federation
Han, Q.-S., Theeboom, M., & Zhu, D. H. (2020) · International Review for the Sociology of Sport · doi:10.1177/1012690220957177 · Cited by 39
Practical Dilemmas for Health Promotion Evaluation
Potvin, L., & McQueen, D. V. (2008) · Springer eBooks · doi:10.1007/978-0-387-79733-5_3 · Cited by 39
From rhetoric to reality: Barriers faced by Health For All initiatives
Berkeley, D., & Springett, J. (2006) · Social Science & Medicine · doi:10.1016/j.socscimed.2005.11.057 · Cited by 39
‘In the eye of the beholder’: perceptions of local impact in English Health Action Zones
Sullivan, H. T., Judge, K., & Sewel, K. (2004) · Social Science & Medicine · doi:10.1016/j.socscimed.2004.02.013 · Cited by 38
Self-citation
Do we really care about unintended outcomes? An analysis of evaluation theory and practice
Jabeen, S. (2016) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2015.12.010 · Cited by 41
The Evaluation of Complex Infrastructure Projects
Gerrits, L., & Verweij, S. (2018) · Edward Elgar Publishing eBooks · doi:10.4337/9781783478422 · Cited by 60
Enabling the transferability of complex interventions: exploring the combination of an intervention’s key functions and implementation
Villeval, M., Bidault, E., Shoveller, J., Alias, F., Basson, J.-C., Frasse, C., Génolini, J., Pons, E., Verbiguié, D., Grosclaude, P., et al. (2016) · International Journal of Public Health · doi:10.1007/s00038-016-0809-9 · Cited by 37
Partnership working and outcomes: do health and social care partnerships deliver for users and carers?
Petch, A., Cook, A., & Miller, E. (2013) · Health & Social Care in the Community · doi:10.1111/hsc.12050 · Cited by 36
Embedding research in health systems: lessons from complexity theory
Caffrey, L., Wolfe, C. D. A., & McKevitt, C. (2016) · Health Research Policy and Systems · doi:10.1186/s12961-016-0128-x · Cited by 36
Transdisciplinary Knowledge Producing Teams: Toward a Complex Systems Perspective
Lotrecchiano, G. R., & Misra, S. (2018) · Informing Science The International Journal of an Emerging Transdiscipline · doi:10.28945/4086 · Cited by 38
Policy evaluation for a complex world: Practical methods and reflections from the UK Centre for the Evaluation of Complexity across the Nexus
Barbrook-Johnson, P., Castellani, B. C., Hills, D., Penn, A. S., & Gilbert, N. (2021) · Evaluation · doi:10.1177/1356389020976491 · Cited by 33
Developing an Evaluation for Crosscutting Policy Goals: The Accessibility Policy Assessment Tool
Bristow, G., Farrington, J. H., Shaw, J., & Richardson, T. (2008) · Environment and Planning A Economy and Space · doi:10.1068/a4092 · Cited by 33
How do policy evaluators understand complexity?
Barbrook-Johnson, P., Proctor, A., Giorgi, S., & Phillipson, J. (2020) · Evaluation · doi:10.1177/1356389020930053 · Cited by 33
Learning from seasoned evaluators: Implications of systems approaches for evaluation practice
Gates, E. (2017) · Evaluation · doi:10.1177/1356389017697613 · Cited by 31
Trois défis pour l'évaluation en promotion de la santé
Potvin, L., Bilodeau, A., & Gendron, S. (2008) · Promotion & Education · doi:10.1177/1025382308093991 · Cited by 30
Evaluating a community-based public health intervention using a complex systems approach
Matheson, A., Walton, M., Gray, R. J., Lindberg, K., Shanthakumar, M., Fyfe, C., Wehipeihana, N., & Borman, B. (2017) · Journal of Public Health · doi:10.1093/pubmed/fdx117 · Cited by 31
Unintended outcomes evaluation approach: A plausible way to evaluate unintended outcomes of social development programmes
Jabeen, S. (2017) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2017.09.005 · Cited by 30
The Meaning of Effectiveness in Assessing Community Initiatives
Schweigert, F. J. (2006) · American Journal of Evaluation · doi:10.1177/1098214006294304 · Cited by 29
Once The Shovel Hits the Ground: Evaluating the Management of Complex Implementation Processes of Public-Private Partnership Infrastructure Projects with Qualitative Comparative AnalysisPreprint
Verweij, S. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2646785 · Cited by 28
A programme theory for liaison mental health services in England
House, A. O., Guthrie, E., Walker, A., Hewsion, J., Trigwell, P. J., Brennan, C. A., Crawford, M., Czoski‐Murray, C., Fossey, M. J., Hulme, C. T., et al. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3539-2 · Cited by 26
Improving health equity through theory-informed evaluations: A look at housing first strategies, cross-sectoral health programs, and prostitution policy
Dunn, J. R., van der Meulen, E., O'Campo, P. J., & Muntaner, C. (2012) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2012.03.006 · Cited by 27
Evaluation of cash transfer programs in sub-Saharan Africa: A methodological review.
Owusu-Addo, E., Renzaho, A., & Smith, B. J. (2018) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2018.02.010 · Cited by 24
L’Outil diagnostique de l’action en partenariat: fondements, élaboration et validation
Bilodeau, A., Galarneau, M., Fournier, M., & Potvin, L. (2011) · Canadian Journal of Public Health · doi:10.1007/bf03404054 · Cited by 26
Measuring the tail of the dog that doesn't bark in the night: the case of the national evaluation of Choose Life (the national strategy and action plan to prevent suicide in Scotland)
Mackenzie, M., Blamey, A., Halliday, E. C., Maxwell, M., McCollam, A., McDaid, D., MacLean, J., Woodhouse, A., & Platt, S. D. (2007) · BMC Public Health · doi:10.1186/1471-2458-7-146 · Cited by 25
Wicked Comparisons: Reflections on Cross-national Research about Health Inequalities in the UK
Blackman, T., Hunter, D. J., Marks, L., Harrington, B., Elliott, E., Williams, G. H., Greene, A., & Mckee, L. (2010) · Evaluation · doi:10.1177/1356389009350016 · Cited by 20
Using the Objective Hermeneutics Method in Policy Evaluation
Mann, S., & Schweiger, J. (2009) · Evaluation · doi:10.1177/1356389009345448 · Cited by 23
The Social Relations of Evaluation
Abma, T. (2006) · SAGE Publications Ltd eBooks · doi:10.4135/9781848608078.n8 · Cited by 23
The complexity of the discussion on effectiveness and evidence in health promotion practices
Bodstein, R. (2007) · Promotion & Education · doi:10.1177/10253823070140010401x · Cited by 21
Capturing complexity in the evaluation of a major area-based initiative in community empowerment: what can a multi-site, multi team, ethnographic approach offer?
Orton, L., Ponsford, R., Egan, M., Halliday, E. C., Whitehead, M. M., & Popay, J. (2019) · Anthropology and Medicine · doi:10.1080/13648470.2018.1508639 · Cited by 21
Thinking with theory as a policy evaluation tool: The case of boarding schools for remote First Nations students
Guenther, J., Benveniste, T., Redman-MacLaren, M. L., Mander, D. J., McCalman, J., O’Bryan, M., Osborne, S., & Stewart, R. (2020) · Evaluation Journal of Australasia · doi:10.1177/1035719x20905056 · Cited by 21
Focusing on Outcomes: Their Role in Partnership Policy and Practice
Petch, A., Cook, A., & Miller, E. (2005) · Journal of Integrated Care · doi:10.1108/14769018200500043 · Cited by 19
Taking stock of complexity in evaluation: A discussion of three recent publications
Gerrits, L., & Verweij, S. (2015) · Evaluation · doi:10.1177/1356389015605204 · Cited by 19
"Healthy Eating - Healthy Action": evaluating New Zealand's obesity prevention strategy
McLean, R. M., Hoek, J. A., Buckley, S., Croxson, B., Cumming, J., Ehau, T. H., Tanuvasa, A. F., Johnston, M., Mann, J. I., & Schofield, G. (2009) · BMC Public Health · doi:10.1186/1471-2458-9-452 · Cited by 18
Towards Modelling Data Infrastructures in the Asset Management Domain
Brous, P., Herder, P., & Janssen, M. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.09.215 · Cited by 18
A Complex Adaptive System of Systems Approach to Human–Automation Interaction in Smart Grid
Fereidunian, A., Lesani, H., Zamani, M. A., Kolarijani, M. A. S., Hassanpour, N., & Mansouri, S. S. (2015) · doi:10.1002/9781119036821.ch12 · Cited by 18
Impact of a community-based pilot intervention to tackle childhood obesity: a ‘whole-system approach’ case study
Gadsby, E. W., Hotham, S., Eida, T., Lawrence, C., & Merritt, R. (2020) · BMC Public Health · doi:10.1186/s12889-020-09694-2 · Cited by 17
‘He called me out of the blue’: An ethnographic exploration of contrasting temporalities in a social prescribing intervention
Gibson, K., Moffatt, S. M., & Pollard, T. M. (2022) · Sociology of Health & Illness · doi:10.1111/1467-9566.13482 · Cited by 16
Impact of a social prescribing intervention in North East England on adults with type 2 diabetes: the SPRING_NE multimethod study
Moffatt, S., Wildman, J., Pollard, T. M., Gibson, K., Wildman, J. M., O’Brien, N., Griffith, B., Morris, S. L., Moloney, E., Jeffries, J. M., et al. (2023) · Public Health Research · doi:10.3310/aqxc8219 · Cited by 17
The Limits of Realist Evaluation
Davis, P. K. (2005) · Evaluation · doi:10.1177/1356389005058476 · Cited by 19
Avaliação da Política Nacional de Promoção da Saúde: perspectivas e desafios
Magalhães, R. (2016) · Ciência & Saúde Coletiva · doi:10.1590/1413-81232015216.07422016 · Cited by 16
Next Generation Data Infrastructures: Towards an Extendable Model of the Asset Management Data Infrastructure as Complex Adaptive System
Brous, P., Janssen, M., & Herder, P. (2019) · Complexity · doi:10.1155/2019/5415828 · Cited by 16
L’efficacité enseignante, quelle modélisation pour servir cette ambition ?
Clanet, J. (2012) · Questions vives recherches en éducation · doi:10.4000/questionsvives.1121 · Cited by 15
System change evaluation: Insights from The Rippel Foundation and its ReThink Health initiative
Gates, E., & Fils‐Aime, F. (2021) · New Directions for Evaluation · doi:10.1002/ev.20462 · Cited by 15

Authors: Barnes, M., Matka, E., & Sullivan, H.
Affiliations: University of Birmingham (GB)
University of the West of England (GB)
Corresponding author: Barnes, Marian
Matka, Elizabeth
Sullivan, Helen
ORCID: Sullivan, Helen
Venue: Evaluation
Publisher: SAGE Publications
Published: 2003-07
Volume: 9
Issue: 3
Pages: 265-284
ISSN: 1356-3890 (ISSN-L)
1461-7153
DOI: 10.1177/13563890030093003
OpenAlex: W2056268972
Cited by: 208
References: 11
Citations by year: 2026: 5 · 2025: 1 · 2024: 4 · 2023: 9 · 2022: 5 · 2021: 8 · 2020: 13 · 2019: 9 · 2018: 13 · 2017: 11 · 2016: 16 · 2015: 14 · 2014: 7 · 2013: 15 · 2012: 11
FWCI: 9.56
Type: journal article
Language: en
Open access: green
eprints.uwe.ac.uk
Licence: cc-by
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Evaluation (publishedVersion)
UWE Research Repository (UWE Bristol) (submittedVersion, cc-by)
University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
University of Brighton Repository (University of Brighton) (submittedVersion)
Topics: Social Sciences › Public Administration › Evaluation and Performance Assessment
Social Sciences › Public Administration › Public Policy and Administration Research
Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
Keywords: Health Action Zones, theory of change, complexity theory, new institutionalism, social constructionism, evaluator role, program development, evidence-based policy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Evidence, Understanding and Complexity: Evaluation in Non-Linear Systems

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks