Cited by 476Evaluation and Performance AssessmentHealth Policy Implementation ScienceAdvanced Causal Inference Techniques

The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.

Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research

Marchal, Bruno, van Belle, Sara, van Olmen, Josefien, Hoerée, Tom, Kegels, Guy·2012·Evaluation

DOI 10.1177/1356389012442444

Open this paper in CoChat — free account

Abstract

This overview aims to stimulate conceptual and practical discussions to help unlock the full potential of realist evaluation in health systems research. Based on a structured literature search, this review maps how the concepts of realist evaluation are applied in health systems research and which methodological problems are encountered. We found a great diversity in the depth of application of the philosophical concepts, use of terminology and scope of application in the research process. Terms of theory-driven evaluation, theories of change and realist evaluation are often used interchangeably. Diverging views exist regarding the nature of ‘mechanism’ and the difference between mechanism and essential context condition. A lack of methodological guidance was highlighted by many authors. Realist evaluation is slowly gaining traction in health systems research, but more clarity is needed concerning the definitions of mechanisms and context and how the configuration of context, mechanism and intervention can be described and assessed.

Integrity signals

Registry

Crossref · 10.1177/1356389012442444

Cites

93 of 93 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Social Theory and Social Structure
Ross, E. J., & Merton, R. Κ. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,318
Social Theory and Social Structure.
Bock, K. E., & Merton, R. Κ. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,687
Systems Thinking, Systems Practice
Checkland, P. B. (1981) · Cumulative Index of Computer Aided Architectural Design · Cited by 6,256
Systems thinking, systems practice
(1982) · European Journal of Operational Research · doi:10.1016/0377-2217(82)90206-5 · Cited by 3,579
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190
A set-theoretic approach to organizational configurations
Fiss, P. C. (2007) · Academy of Management Review · doi:10.5465/amr.2007.26586092 · Cited by 2,225
Evidence-based Policy
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
Critical realism: essential readings
(1999) · Choice Reviews Online · doi:10.5860/choice.36-5017 · Cited by 1,683
Designing and evaluating complex interventions to improve health care
Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L. (2007) · BMJ · doi:10.1136/bmj.39108.379965.be · Cited by 1,331
Theory-driven evaluations
Chen, H.-T. (1990) · Medical Entomology and Zoology · Cited by 1,227

Show all 92 references

Research works printed but not linked: 4

Connell JP, 1995, New Approaches To Evaluating Community Initiatives. Concepts, Methods, and Contexts
Kernick D, 2005, British Journal of Medical Practice, 55, 908
Pawson R, 1997, Realistic Evaluation
Weiss C, 1995, New Approaches to Evaluating Comprehensive Community Initiatives: Concepts, Methods and Contexts

Sorted by citations

Items 201–300 of 474

In or out?: Exploring selection processes of farmers in cocoa sustainability standards and certification programmes in Ghana
Skalidou, D. (2019) · UEA Digital Repository (University of East Anglia) · doi:10.13140/rg.2.2.28002.58560 · Cited by 15
Exploring the Potential of Theory-Based Evaluation to Strengthen Marine Spatial Planning PracticePreprint
Zuercher, R., Motzer, N., Ban, N. C., Flannery, W., Guerry, A. D., Magris, R. A., Mahajan, S. L., Spalding, A. K., Stelzenmüller, V., & Kramer, J. G. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4297861 · Cited by 13
A systematic realist synthesis of digital interventions for enhancing mental health at work: contexts, mechanisms, and outcomes
Zuberbühler, J. P., Pietrantoni, L., Mazzetti, G., Angelis, M. D., Giusino, D., Román-Niaves, M. S., Guglielmi, D., & Salanova, M. (2025) · International Journal of Mental Health Systems · doi:10.1186/s13033-024-00655-5 · Cited by 20
Realist Evaluation of the “Abiye” Safe Motherhood Initiative in Nigeria: Unveiling the Black-Box of Program Implementation and Health System Strengthening
Oladimeji, O. J., & Fatusi, A. O. (2022) · Frontiers in Health Services · doi:10.3389/frhs.2022.779130 · Cited by 13
Improving mental ill-health with psycho-social group interventions in South Asia–A scoping review using a realist lens
Mathias, K. R., Jain, S., Fraser, R., Davis, M., Kimijima-Dennemeyer, R. M., Pillai, P., Deshpande, S. N., & Wolters, M. K. (2023) · PLOS Global Public Health · doi:10.1371/journal.pgph.0001736 · Cited by 16
A realist evaluation of the implementation of a national tobacco control programme and policy in India
Hebbar, P. B., Dsouza, V., Nagelhout, G. E., Belle, S. V., Srinivas, P. N., van Schayck, O. C. P., Babu, G. R., & Bhojani, U. M. (2024) · Health Policy and Planning · doi:10.1093/heapol/czae081 · Cited by 12
Stakeholder involvement in realist evaluation: A scoping review and best fit framework synthesis
Malengreaux, S., Martens, M., Pleguezuelo, V. C., Ibiloye, O., Belle, S. V., Renmans, D., & Belrhiti, Z. (2024) · Evaluation · doi:10.1177/13563890241279562 · Cited by 12
“While RE is still a relatively new approach, it is becoming increasingly relevant in the fields of evaluation, and more specifically in health policy and system research (Marchal et al., 2012; Van Belle, 2022).”
Collaboration for Developing and Sustaining Community Dementia-Friendly Initiatives: A Realist Evaluation
Thijssen, M., Graff, M. J. L., Lexis, M. A. S., der Sanden, M. W. G. N., Radford, K., Logan, P., Daniëls, R., & Kuijer‐Siebelink, W. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20054006 · Cited by 12
Using realist evaluation to assess primary healthcare teams’ responses to intimate partner violence in Spain
Goicolea, I., Hurtig, A., Sebastiån, M. S., Marchal, B., & Vives‐Cases, C. (2015) · Gaceta Sanitaria · doi:10.1016/j.gaceta.2015.08.005 · Cited by 14
Self-citation
Evaluating program effects: Conceptualizing and demonstrating a typology
Gao, X., Shen, J., Wu, H., & Krenn, H. Y. (2018) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2018.10.008 · Cited by 12
Categories of context in realist evaluation
Sheaff, R., Doran, N., Harris, M., Lang, I., Medina-Lara, A., Fornasiero, M., Ball, S., McGregor-Harper, J., & Bethune, R. M. (2021) · Evaluation · doi:10.1177/1356389020968578 · Cited by 12
“The definitional boundary between mechanisms and context is often unclear, even disputed (Astbury and Leeuw, 2010; Marchal et al., 2012; Rog, 2012).”
Background
Schools’ engagement with the Get Set London 2012 Olympic education programme: Empirical insights from schools in a non-hosting region
Chen, S., & Henry, I. P. (2017) · European Physical Education Review · doi:10.1177/1356336x17721437 · Cited by 12
“The last 15 years have seen a gradual increase in the number of papers applying realist evaluation principles (Marchal et al., 2012), for example in the contexts of policy, practice, and other social evaluation (Gill and Turbin, 1999; Greenhalgh et al.”
Background
Capturing the realities of sports programmes: systematic ‘messiness’?
Daniels, J., Bell, B. J., & Horrocks, C. (2018) · International Journal of Sport Policy and Politics · doi:10.1080/19406940.2018.1513414 · Cited by 12
How do participatory methods shape policy? Applying a realist approach to the formulation of a new tuberculosis policy in Georgia
Marchal, B., Abejirinde, I. O., Sulaberidze, L., Chikovani, I., Uchaneishvili, M., Shengelia, N., Diaconu, K., Vassall, A., D, A. Z. M., Giralt, A. N., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047948 · Cited by 13
Self-citation
Vignettes to Support Theory Refinement: Methodological Insights From a Realist Evaluation
Aldamman, K., Vallières, D. F., & Gilmore, B. (2024) · International Journal of Qualitative Methods · doi:10.1177/16094069231216607 · Cited by 11
“Notably, researchers have noted dif ﬁ culties distinguishing between Mechanisms and Contexts Fields (Pedersen, et al., 2012) and uncovering either throughout data collection (Marchal et al., 2012).”
Protocol for a mixed methods realist evaluation of regional District Health Board groupings in New Zealand
Stokes, T., Atmore, C., Penno, E., Richard, L., Wyeth, E., Richards, R., Doolan‐Noble, F., Gray, A. R., Sullivan, T. A., & Gauld, R. D. C. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030076 · Cited by 11
SEEP-CI: A Structured Economic Evaluation Process for Complex Health System Interventions
Madan, J. J., Kumar, M. B., Taegtmeyer, M., Barasa, E., & Singh, S. P. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17186780 · Cited by 11
“These challenges have led to the development of approaches such as realist evaluation, which explicitly considers the mechanisms and context in which an intervention acts [24], and has proven to be valuable for health systems research [25].”
Background
Building resilience in oncology teams: Protocol for a realist evaluation of multiple cases
Tremblay, D., Touati, N., Kilpatrick, K., Durand, M., Turcotte, A., Prady, C. C. M., Poder, T. G., Richard, P. O., Soldera, S. V., Berbiche, D., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0268393 · Cited by 11
“This approach is strategic as it can have a positive influence on the adoption of new practices based on research findings, and can contribute to solving complex problems [72, 114].”
Background
Adoption and Initial Implementation of a National Integrated Care Programme for Diabetes: A Realist Evaluation
O’Neill, K., Riordan, F., Racine, E., Tracey, M., Papoutsi, C., Kearney, P. M., & McHugh, S. (2022) · International Journal of Integrated Care · doi:10.5334/ijic.5815 · Cited by 11
Effective health care for older people resident in care homes: the optimal study protocol for realist review
Goodman, C. M., Gordon, A. L., Martin, F. C., Davies, S. L., Iliffe, S., Bowman, C., Schneider, J. M., Meyer, J. E., Victor, C. R., Gage, H., et al. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-49 · Cited by 11
“They will provide an account of the behaviour and perceived interrelationships of the processes that are responsible for achieving change [33] and operational definitions to inform how they may be recognised and tracked within different models of service delivery to care homes.”
Background
The Troubled Families Programme: Learning about policy impact through realist case study research
Parr, S., & Churchill, H. (2019) · Social Policy and Administration · doi:10.1111/spol.12529 · Cited by 16
The use of realist approaches for health research in Indigenous communities
Bergeron, D. A., Tremblay, M., Dogba, M. J., Martin, D. H., & McGavock, J. (2021) · AlterNative An International Journal of Indigenous Peoples · doi:10.1177/1177180121996063 · Cited by 11
“…may be difficult to operationalize (Robert & Ridde, 2013) given that there are some difficulties in conceptualizing and differentiating some concepts used in RAs such as elements of context and mechanisms (Astbury & Leeuw, 2010; Marchal et al., 2012; Robert & Ridde, 2013; Salter & Kothari, 2014).”
Background
Advancing socio-ecological considerations in impact assessment of extractive industries: A realist interview study in the Canadian context
Brown, J. A., Lewans, M., Menon, D., & Nykiforuk, C. I. J. (2024) · Environmental Science & Policy · doi:10.1016/j.envsci.2024.103677 · Cited by 11
Realist evaluation for programs and services in the health area: an integrative review of the theoretical and methodological literature
Quintans, J. R., Yonekura, T., Trapé, C. A., & Soares, C. B. (2020) · Revista Latino-Americana de Enfermagem · doi:10.1590/1518-8345.3933.3255 · Cited by 10
“The authors recommend greater clarity regarding the definitions of mechanisms and context, two elements considered structural in the realist evaluation, in addition to the outcome(8).”
BackgroundMethodInfluential
A realist evaluation of a home-based end of life care service for children and families: what works, for whom, how, in what circumstances and why?
Malcolm, C. A., & Knighting, K. (2022) · BMC Palliative Care · doi:10.1186/s12904-022-00921-8 · Cited by 10
Testing the effectiveness and acceptability of online supportive supervision for mental health practitioners in humanitarian settings: a study protocol for the caring for carers project
Wells, R., Acartürk, C., Mozumder, M. K., Kurt, G., Klein, L. A., Lekkeh, S. A., Beetar, A., Jahan, S., Almeamari, F., Faruk, M. O., et al. (2023) · BMC Psychiatry · doi:10.1186/s12888-023-05246-1 · Cited by 10
“Combining routine qualitative and quantitative data collection and analysis, the RE framework examines processes and mechanisms underpinning the intervention [15, 16].”
BackgroundMethod
Identifying the factors that influence teacher practice change in a single case study
Forrest, R. H.-J., Lowe, R., Potts, M. A., & Poyser, C. (2019) · Educational Psychology in Practice · doi:10.1080/02667363.2019.1623761 · Cited by 10
“As Marchal et al. (2012) state, “defining ‘context’ and separating ‘mechanism’ from ‘context’ remains a difficult issue” (p. 207).”
‘Function First’: how to promote physical activity and physical function in people with long-term conditions managed in primary care? A study combining realist and co-design methods
Law, R., Langley, J., Hall, B., Burton, C. R., Hiscock, J., Williams, L., Morrison, V., LEMMEY, A. B., Lovell-Smith, C., Gallanders, J., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-046751 · Cited by 9
A Realist-Informed Review of Digital Empowerment Strategies for Adolescents to Improve Their Sexual and Reproductive Health and Well-being
Goh, K., Contractor, S. Q., & Belle, S. V. (2022) · Journal of Urban Health · doi:10.1007/s11524-022-00678-8 · Cited by 17
Realist synthesis of a rapid response system in managing mental state deterioration in acute hospital settings
Dziruni, T. B., Hutchinson, A. M., Coomer, J., Keppich‐Arnold, S., & Bucknall, T. K. (2024) · International Journal of Mental Health Nursing · doi:10.1111/inm.13347 · Cited by 10
Hospital to community in Wales: What is the value of optometrists playing a greater role in managing neovascular AMD and glaucoma in primary care?
Ryan, B., Jones, M., Anderson, P., Reynolds, R., Nicholls, R., Cullen, K., Davies, M. W., North, R. V., Molik, B., & Wallace, C. A. (2024) · Ophthalmic and Physiological Optics · doi:10.1111/opo.13397 · Cited by 9
“27 Cycle 2 further dimensionalised the embryonic Initial Programme Theory—iteratively confirming, refuting and refining the elements through a systematic review of the literature.”
The work integrated learning experience of a university student with intellectual disability: a descriptive case study
Rillotta, F., Lindsay, L., & Gibson-Pope, C. (2021) · International Journal of Inclusive Education · doi:10.1080/13603116.2021.1937343 · Cited by 9
“This study also explored whether the UTHP placement can be functional in other contexts (Marchal et al. 2012; Pawson and Tilley 1997).”
Background
Is self-regulation key in reducing running-related injuries and chronic fatigue? A randomized controlled trial among long-distance runners
van Iperen, L. P., JONGE, J. D., Gevers, J., Vos, S. B., & Hespanhol, L. (2022) · Journal of Applied Sport Psychology · doi:10.1080/10413200.2021.2015479 · Cited by 10
“In addition, mixed methods and “realist approaches” may allow one to detect patterns otherwise missed out on, particularly in combination with more systematically gathered qualitative information (see Marchal et al., 2012; Ryba et al., 2020).”
Background
How to develop a realist programme theory using Margaret Archer’s structure–agency–culture framework: The case of adolescent accountability for sexual and reproductive health in urban resource-constrained settings
Belle, S. V., Abejirinde, I. O., Ssennyonjo, A., Srinivas, P. N., Hebbar, P. B., & Marchal, B. (2023) · Evaluation · doi:10.1177/13563890231185167 · Cited by 9
“It may help to involve social scientists who are knowledgeable about a broad range of social phenomena, paradigms and theories, and to embed the evaluation in a broader cycle of theory construction (Greenhalgh and Emmel, 2018; Jones, 2018; Marchal et al., 2012).”
BackgroundSelf-citation
The role of provincial health administration in supporting district health management teams in the Democratic Republic of Congo: eliciting an initial programme theory of a realist evaluation
Bosongo, S., Belrhiti, Z., Chenge, F. M., Criel, B., Coppieters, Y., & Marchal, B. (2024) · Health Research Policy and Systems · doi:10.1186/s12961-024-01115-9 · Cited by 11
“In practice, RE begins and ends with a theory [27, 28].”
BackgroundSelf-citation
Untitled work
Hjort (2021) · doi:10.1136/bmjopen-2020-042728 · Cited by 11
How can we strengthen mental health services in Swedish youth clinics? A health policy and systems study protocol
Sundberg, L. R., Christianson, M., Wiklund, M., Hurtig, A., & Goicolea, I. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-048922 · Cited by 12
Context, mechanisms and outcomes of dementia special care units: An initial programme theory based on realist methodology
Palm, R., Fahsold, A., Roes, M., & Holle, B. (2021) · PLoS ONE · doi:10.1371/journal.pone.0259496 · Cited by 9
“Developed in the 1990s by Pawson & Tilly in the UK [19], the realist methodology is gaining increasing attention in nursing research [20] and other health care research fields [21].”
Background
How to research the mechanisms of non-pharmacological cardiac interventions
Clark, A. M., Thirsk, L. M., Wiens, K. S., Ski, C. F., & Thompson, D. R. (2015) · International Journal of Cardiology · doi:10.1016/j.ijcard.2015.08.114 · Cited by 9
Aid effectiveness and programmatic effectiveness: a proposed framework for comparative evaluation of different aid interventions in a particular health system
Haque, H. M. J., Hill, P. C., & Gauld, R. D. C. (2017) · Global Health Research and Policy · doi:10.1186/s41256-017-0029-8 · Cited by 9
“[31], realist evaluation has been increasingly applied in a variety of fields within health systems research.”
Background
Protocol for an implementation and realist evaluation of a new organisational model for primary care practices in the canton of Vaud, Switzerland
Leuthold, M. S., Schwarz, J., Marti, J., Perraudin, C., Hudon, C., Peytremann‐Bridevaux, I., Senn, N., & Cohidon, C. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040154 · Cited by 9
Correction to: The patient advisor, an organizational resource as a lever for an enhanced oncology patient experience (PAROLEonco): a longitudinal multiple case study protocol
Pomey, M., de Guise, M., Desforges, M., Bouchard, K., Vialaron, C., Normandin, L., Nelea, M. I., Fortin, I., Ganache, I., Régis, C., et al. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06065-4 · Cited by 9
Moving on from logical frameworks to find the ‘missing middle’ in international development programmes
Floate, H. J., Durham, J., & Marks, G. C. (2018) · Journal of Development Effectiveness · doi:10.1080/19439342.2018.1551921 · Cited by 10
“CMO configurations are identified in the analysis phase of the realist evaluation cycle and are translated through a synthesis of evidence into a middle-range theory, which is then tested and retested iteratively until an end theory is reached (Marchal et al. 2012).”
Background
Qualitative instruments involving clients as co-researchers to assess and improve the quality of care relationships in long-term care: an evaluation of instruments to enhance client participation in quality research
Scheffelaar, A., Bos, N., Triemstra, M., de Jong, M., Luijkx, K. G., & van Dulmen, S. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-033034 · Cited by 8
Unpacking Mechanisms in Climate Resilient Agriculture Interventions
Barrett, S., D’Errico, S., Anderson, S., & Nebsu, B. (2020) · New Directions for Evaluation · doi:10.1002/ev.20423 · Cited by 8
“When realist evaluation is applied empirically, it is typically within the field of health systems (Marchal et al., 2012).”
Method
A Realist Synthesis of Community-Based Interventions in Vector-Borne Diseases
Pérez, D., Robert, É., Pérez, E. J., Vanlerberghe, V., Lefèvre, P., & Ridde, V. (2021) · American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.20-0944 · Cited by 8
“difficulties differentiating them from activities and specific intervention resources has also been reported in the literature.(14,72,73) However, investigators do not necessarily need an understanding of mechanisms to conduct research”
Background
‘She taught me’: factors consumers find important in nurse practitioner and pharmacist prescriber services
Officer, T. N., Cumming, J., & McBride‐Henry, K. (2021) · Human Resources for Health · doi:10.1186/s12960-021-00587-y · Cited by 12
Clinical and healthcare improvement through My Health Record usage and education in general practice (CHIME-GP): a study protocol for a cluster-randomised controlled trial
Bonney, A. D., Metusela, C., Mullan, J., Barnett, S., Rhee, J., Kobel, C., & Batterham, M. J. (2021) · Trials · doi:10.1186/s13063-021-05438-8 · Cited by 8
Realist evaluation of health promotion interventions: a scoping review
Malengreaux, S., Doumont, D., Scheen, B., Durme, T. V., & Aujoulat, I. (2022) · Health Promotion International · doi:10.1093/heapro/daac136 · Cited by 8
A community pharmacy practice-based research network: A protocol for a realist evaluation
Handyside, L., Cameron, J., & Freeman, C. R. (2022) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2022.12.002 · Cited by 8
Exploring the experiences of people living with dementia in Dementia Friendly Communities (DFCs) in Northern Ireland: a realist evaluation protocol
Craig, S. G., Mitchell, G., O’Halloran, P. D., Stark, P., & Wilson, C. B. (2023) · BMC Geriatrics · doi:10.1186/s12877-023-04090-y · Cited by 7
“This is a theory-driven methodology that supports researchers in evaluating social programmes, such as DFCs [19].”
Method
Realist synthesis protocol on the effectiveness of a rapid response system in managing mental state deterioration in acute hospital settings
Dziruni, T. B., Hutchinson, A. M., Keppich‐Arnold, S., & Bucknall, T. K. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-077597 · Cited by 8
“28 29 The methodology is based on the philosophical paradigm of realism, which positions itself between the positivist and the constructivist paradigms and emphasises the search for alignment between reality and our constructions of reality.”
Method
Designing and evaluating provider results-based financing for tuberculosis care in Georgia: a realist evaluation protocol
Marchal, B., Giralt, A. N., Sulaberidze, L., Chikovani, I., & Abejirinde, I. O. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030257 · Cited by 7
Self-citation
Promoting physical activity and physical function in people with long-term conditions in primary care: the Function First realist synthesis with co-design
Law, R., Langley, J., Hall, B., Burton, C. R., Hiscock, J., Williams, L., Morrison, V., LEMMEY, A. B., Lovell-Smith, C., Gallanders, J., et al. (2021) · Health Services and Delivery Research · doi:10.3310/hsdr09160 · Cited by 8
At the interface between the formal and informal, the actual and the real: a realist study protocol for governance and accountability practices in urban settings focusing on adolescent sexual and reproductive health and rights
Belle, S. V. (2022) · International Journal for Equity in Health · doi:10.1186/s12939-022-01644-1 · Cited by 7
A personal reflection on co-creation in public health: a dream partly realised
Bauman, A. E. (2022) · Public Health Research & Practice · doi:10.17061/phrp3222210 · Cited by 7
Standing back and looking forward: Editors’ reflections on the 20th Anniversary of Evaluation
Stern, E., Saunders, M., & Stame, N. (2015) · Evaluation · doi:10.1177/1356389015608757 · Cited by 7
Urban Regeneration in Glasgow: Looking to the Past to Build the Future? The Case of the ‘New Gorbals’
Clark, J. D., & Wright, V. (2018) · The urban book series · doi:10.1007/978-3-319-72311-2_3 · Cited by 7
What key conditions and mechanisms generate health inequalities research in different contexts? Study protocol for two realist explanatory case studies
Cash‐Gibson, L., Martínez‐Herrera, E., & Benach, J. (2021) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2021.101986 · Cited by 7
What can realist evaluation tell us about how coaching interventions work?
Kovács, L. C., & Corrie, S. (2016) · The Coaching Psychologist · doi:10.53841/bpstcp.2016.12.2.59 · Cited by 7
Addressing the unmet contraceptive need of adolescents and unmarried youth: Act or Interact Learning from comprehensive interventions in China and Latin America
Decat, P. (2015) · Ghent University Academic Bibliography (Ghent University) · doi:10.13140/rg.2.1.3150.3843 · Cited by 7
Sport and exercise-based interventions for young people in alternative provisions ‘what, where, how, and for who’: a realist review
Simmons, T., Brinkley, A., Quarmby, T., Horner, L., Sherar, L. B., Hooper, O., & Kinnafick, F. (2024) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2024.2303773 · Cited by 6
“Moreover, a realist approach can develop potential explanations for what has been found within the literature and create theories – known as transferable theories (see Merton, 1967) – which can be transferred to inform programme implementation in di ﬀ ering environments (Marchal et al., 2012).”
Next of kin participation in the care of older persons in nursing homes: A pre–post non-randomised educational evaluation, using within-group and individual person-level comparisons
Westergren, A., Ahlström, G., Persson, M., & Behm, L. (2021) · PLoS ONE · doi:10.1371/journal.pone.0244600 · Cited by 5
Realist evaluation and its role in the stages of explanatory research based on critical realism
Parra, J. D. (2023) · Journal of Critical Realism · doi:10.1080/14767430.2023.2251816 · Cited by 6
“Taking into account previous cautions expressed by RE practitioners (Greenhalgh et al. 2015; Marchal et al. 2012), I acknowledge the validity of critics ’ concerns that applied RE might have yielded to the temptation of excessively focusing on the analysis of ‘ actual measurable events and…”
A realist evaluation protocol: assessing the effectiveness of a rapid response team model for mental state deterioration in acute hospitals
Dziruni, T. B., Hutchinson, A. M., Keppich‐Arnold, S., & Bucknall, T. K. (2024) · Frontiers in Health Services · doi:10.3389/frhs.2024.1400060 · Cited by 6
Confidentiality, anonymity and amnesty for midwives in distress seeking online support – Ethical?
Pezaro, S., Clyne, W., & Gerada, C. (2016) · Nursing Ethics · doi:10.1177/0969733016654315 · Cited by 10
Avaliação de programas, estratégias e ações de saúde: um diálogo com o realismo crítico
Costa, D. M., & Magalhães, R. (2019) · Saúde em Debate · doi:10.1590/0103-11042019s715 · Cited by 6
Understanding the impact of a residential housing programme for people living with HIV/AIDS: a realist evaluation protocol
Fletcher-Hildebrand, S., Alimezelli, H. T., Carr, T., Lawson, K. L., Ali, A., & Groot, G. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-044522 · Cited by 8
“72 For example, the iterative nature of realist evaluations may inhibit forward movement in the evaluation.”
Background
The logic behind the initiatives of national governing bodies in Flanders to improve organised youth sport: A theory-based evaluation approach
Ricour, M., Bosscher, V. D., & Shibli, S. (2023) · International Journal of Sport Policy and Politics · doi:10.1080/19406940.2022.2161595 · Cited by 6
“…of organised youth sport and decide whether and how to stimulate demand in their affiliated sport clubs through (a) making their services more available to attract new youth members and/or (b) improving the quality of their supply for existing youth members (Bullough 2011, Meldrum & McDonald 1995).”
Background
Longitudinal realist evaluation of the Dementia PersonAlised Care Team (D-PACT) intervention: protocol
Wheat, H. C., Weston, L., Oh, T. M., Morgan‐Trimmer, S., Ingram, W. M., Griffiths, S., Sheaff, R., Clarkson, P., Medina‐Lara, A., Musicha, C., et al. (2023) · BJGP Open · doi:10.3399/bjgpo.2023.0019 · Cited by 9
National Rural Health Mission reforms in light of decentralised planning in Kerala, India: a realist analysis of data from three witness seminars
Hari Sankar, D., Benny, G., Jaya, S., & Nambiar, D. (2024) · BMC Public Health · doi:10.1186/s12889-024-18181-x · Cited by 6
“Realist approaches have been used to look at complex health systems phenomena and qualitative research [24, 25], as well as community action for health in particular [26–28], and were considered uniquely appropriate for our analysis.”
Method
How, why, for whom and in what context, do sexual health clinics provide an environment for safe and supported disclosure of sexual violence: protocol for a realist review
Caswell, R. J., Maidment, I., Ross, J. D. C., & Bradbury‐Jones, C. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-037599 · Cited by 7
Are We Using the Right Approach to Change Newborn Care Practices in the Community? Qualitative Evidence From Ethiopia and Northern Nigeria
Hill, Z. E., Scheelbeek, P., Hamza, Y. A., Amare, Y., & Schellenberg, J. (2020) · Global Health Science and Practice · doi:10.9745/ghsp-d-19-00410 · Cited by 6
A Realist Evaluation of Team Interventions in Acute Hospital Contexts—Use of Two Case Studies to Test Initial Programme Theories
Cunningham, U., De Brún, A., Willgerodt, M., Abu-Rish Blakeney, E., & McAuliffe, E. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168604 · Cited by 5
Evaluation of a Capacity-Building Programme of District Health Managers in India: A Contextualised Theoretical FrameworkPreprint
Srinivas, P. N., Marchal, B., Macq, J., Kegels, G., & Criel, B. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2451132 · Cited by 6
Self-citation
Non-dispensing pharmacists integrated into general practices as a new interprofessional model: a qualitative evaluation of general practitioners’ experiences and views
Hazen, A. C. M., Sloeserwij, V., GROOT, E. D., de Gier, J. J., de Wit, N. J., Bont, A. A. D., & Zwart, D. L. M. (2024) · BMC Health Services Research · doi:10.1186/s12913-024-10703-y · Cited by 5
“…external to the intervention, present or occurring even if the intervention does not lead to an outcome, and which may have influence on the outcome” [23] and mechanism-elements as underlying processes or structures, usually hidden, which operate in particular contexts and generate outcomes [24].”
Background
How did New Zealand’s regional District Health Board groupings work to improve service integration and health outcomes: a realist evaluation
Penno, E., Atmore, C., Maclennan, B., Richard, L., Wyeth, E., Richards, R., Doolan‐Noble, F., Gray, A. R., Sullivan, T. A., Gauld, R. D. C., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-079268 · Cited by 5
“Using a realist-informed evaluation, we sought to understand how NZ’s four regional DHB groupings worked to improve health system integration, health outcomes and equity for their respective regional populations.”
Background
Guiding Through Turbulent Times: Coaching During Merger and Acquisition
Schmitt, J., Vasquez, C., & Pauknerová, D. (2024) · Journal of Change Management · doi:10.1080/14697017.2024.2361415 · Cited by 5
“The outcomes (O) explored focus on the effects on the employees’ mental health, more specifically, how the interventions changed how people work, think and behave to better cope with the challenges they are facing (Marchal et al., 2012).”
Using Realistic Evaluation to understand how interventions work on dairy farms
Koralesky, K. E., Mills, K. E., von Keyserlingk, M. A. G., & Weary, D. M. (2021) · animal · doi:10.1016/j.animal.2021.100233 · Cited by 5
Realist evaluation of the impact of the research translation process on health system sustainability: a study protocol
Mosedale, A., Hendrie, D. V., Geelhoed, E. A., Zurynski, Y. A., & Robinson, S. (2022) · BMJ Open · doi:10.1136/bmjopen-2020-045172 · Cited by 5
Generating program theories for a trauma consultancy service in early learning settings: Insights on using realist methodology
Blewitt, C., Bajayo, R., Cameron, L., Sun, Y., Morris, H., & Skouteris, H. (2023) · Australasian Journal of Early Childhood · doi:10.1177/18369391221146630 · Cited by 5
“However, researchers report difficulty applying the methodology in practice (Dalkin et al., 2015; Marchal et al., 2012).”
Background
How benchmarking motivates colostrum management practices on dairy farms: A realistic evaluation
Wilson, D. J., Roche, S. M., Pempek, J. A., Habing, G. G., Proudfoot, K. L., & Renaud, D. L. (2023) · Journal of Dairy Science · doi:10.3168/jds.2023-23383 · Cited by 5
A realist evaluation of policy interventions to reduce public subsidies of private dental care in Finland
Vesinurm, M., Halminen, O., Linna, M., Mikkola, H., & Lillrank, P. M. (2023) · Community Dentistry And Oral Epidemiology · doi:10.1111/cdoe.12938 · Cited by 5
How do medical students learn about SDH in the community? A qualitative study with a realist approach
Haruta, J., Takayashiki, A., Ozone, S., Maeno, T., & Maeno, T. (2022) · Medical Teacher · doi:10.1080/0142159x.2022.2072282 · Cited by 5
“The realist approach has been offered in several theory-based evaluations developed within the social sciences (Pawson et al. 1997; Marchal et al. 2012; Wong et al. 2016).”
Background
Facebook as a Medium for the Support and Enhancement of Ambulatory Care for People With Diabetes: Qualitative Realist Evaluation of a Real-World Trial
Cleal, B. R., Willaing, I., Høybye, M. T., & Thomsen, H. H. (2020) · JMIR Diabetes · doi:10.2196/18146 · Cited by 5
“Theory-driven evaluation represents an ideal approach to the appraisal of complex real-world interventions [30].”
Background
The need for theory-based evaluation of care coordination initiatives: Considerations from the 2017 International Conference on Realist Research, Evaluation and Synthesis
Vugts, M. A. P., Liu, H., Boumans, J., & Boydell, E. (2018) · International Journal of Care Coordination · doi:10.1177/2053434518779751 · Cited by 5
“using “normal” RCT methods for the aims of RE) may be an oxymoron because realistic conceptions on generative causation and complexity contradict the assumption that control groups are necessary in experimentation, that random allocation results in equal group conditions in all relevant aspects, and that quantitative…”
Background
An integrated realist evaluation model to evaluate organisational interventions
Roodbari, H., Nielsen, K. M., & Axtell, C. M. (2021) · Academy of Management Proceedings · doi:10.5465/ambpp.2021.10830abstract · Cited by 5
Champions as catalysts for change: a realist evaluation of their impact in primary care
Martin, É., Bergeron, D. A., & Gaboury, I. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13573-0 · Cited by 4
“Emerging themes were organized in the form of Context, Mechanism, or Outcome, and then regrouped as CMO configurations for each case [60–62].”
Conversations in Evaluation – Exploring 30 years of debates in our field
Delahais, T., & Thyrard, A. (2025) · Evaluation · doi:10.1177/13563890251406246 · Cited by 4
“This is the reason why this group of articles is also much more connected to other evaluation journals, especially AJE and NDE , and also, although this is not visible from our data, with public health (see, for example, Marchal et al., 2012).”
How to do (or not to do) realist evaluations to advance theory, practice, and justice in health systems research
Aivalli, P., Hebbar, P. B., Mbachu, C. O., Srinivas, P. N., Abímbọ́lá, Ṣ., & Dada, S. (2025) · Health Policy and Planning · doi:10.1093/heapol/czaf080 · Cited by 5
“…applied to explore intersectoral collaboration, community engagement, and health policy implementation in a range of contexts and settings (Marchal et al. 2012, Jagosh et al. 2015, Bergeron et al. 2017, Adhikari et al. 2019, Irvine Fitzpatrick et al. 2021, Mbachu et al. 2022, Hogeling et…”
Influential
A Workplace Intervention to Strengthen Supervisor Support for Employees With Common Mental Health Problems: A Mixed-Methods Realist Evaluation
van Hees, S., Carlier, B. E., Joosen, M. C. W., Blonk, R. W. B., & Oomens, S. P. C. J. (2024) · Scandinavian Journal of Work and Organizational Psychology · doi:10.16993/sjwop.249 · Cited by 4
A comparative evaluation of 23 projects on mental health and wellbeing for veterans and first responders
Lukersmith, S., Woods, C., Salvador‐Carulla, L., Niyonsenga, T., Mohanty, I., Gutiérrez-Colosía, M. R., Díaz-Milanés, D., GARCÍA-ALONSO, C. R., & Büsst, C. J. (2025) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2025.152599 · Cited by 4
Capacity building of district health management teams in the era of provincial health administration reform in the Democratic Republic of Congo: a realist evaluation protocol
Bosongo, S., Belrhiti, Z., Chenge, F., Criel, B., & Marchal, B. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073508 · Cited by 4
Self-citation
On Smart Cities and Triple-Helix Intermediaries: A Critical-Realist Perspective
Chatzinikolaou, D. (2025) · Smart Cities · doi:10.3390/smartcities8030074 · Cited by 9
“Although a few publications implement literature review approaches based on the lens of critical realism (cf. [64]), it remains uncommon to see research explicitly embracing and applying this philosophical stance in smart-city scholarship (e.g., [65,66]) or research policy (e.g., [67]).”
Developing evaluation capacities in integrated care projects: Lessons from a scientific support mission implemented in Belgium
Charlier, N., Colman, E., Irusta, L. A., Anthierens, S., Durme, T. V., Macq, J., & Pétré, B. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.958168 · Cited by 4
“The theory of change “describes how a program brings about specific long-term outcomes through a logical sequence of intermediate outcomes (and actions)” (32), and it is embedded in the realist evaluation paradigm (33) adopted by FAITH.”
Background
Mechanisms fostering the sustainability of actualized value propositions of implemented e‐Government projects in Sub‐Saharan Africa: A realist evaluation case of Nigeria and Rwanda
Chidama, Y. E., & Ononiwu, C. (2022) · The Electronic Journal of Information Systems in Developing Countries · doi:10.1002/isd2.12256 · Cited by 6
Recruiting residents from disadvantaged neighbourhoods for community‐based health promotion and disease prevention services in Denmark—How, why and under what circumstances does an active door‐to‐door recruitment approach work?
Bysted, S., Overgaard, C., Simpson, S. A., Curtis, T., & Bøggild, H. (2021) · Health & Social Care in the Community · doi:10.1111/hsc.13246 · Cited by 4
Standardised Data on Initiatives – STARDIT: Beta VersionPreprint
Nunn, J., Shafee, T. M. A., Chang, S., Stephens, R., Elliot, J., Oliver, S. R., John, D., Smith, M. P., Orr, N., Preston, J., et al. (2021) · Research Square · doi:10.21203/rs.3.rs-934285/v1 · Cited by 4
Implementation of large-system transformation initiatives in the New Zealand health system
Sharma, K. (2021) · doi:10.26686/wgtn.15130413.v1 · Cited by 4
Unravelling mechanisms underlying the action principles of a community-based health promotion programme: a realist evaluation
de Jong, M. A. J. G., Wink, G., Koelen, M. A., & Wagemakers, A. (2023) · Archives of Public Health · doi:10.1186/s13690-023-01027-0 · Cited by 3
“Step 1 Transcripts of the evaluation sessions (Sources I–III, Table 1) were coded in terms of context conditions (C), underlying mechanisms (M) in the actual programme, and outcomes observed by respondents (O) [58].”
Background

Authors: Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G.
Affiliations: ITM-A, Belgium
BTC, Belgium
ORCID: Marchal, Bruno
Kegels, Guy
Venue: Evaluation
Publisher: SAGE Publications
Published: 2012-04
Volume: 18
Issue: 2
Pages: 192-212
ISSN: 1356-3890 (ISSN-L)
1461-7153
DOI: 10.1177/1356389012442444
OpenAlex: W2077589521
Cited by: 476
References: 92
Citations by year: 2026: 22 · 2025: 29 · 2024: 41 · 2023: 29 · 2022: 51 · 2021: 64 · 2020: 31 · 2019: 40 · 2018: 37 · 2017: 28 · 2016: 31 · 2015: 34 · 2014: 24 · 2013: 12 · 2012: 3
FWCI: 30.83
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Evaluation (publishedVersion)
Topics: Social Sciences › Public Administration › Evaluation and Performance Assessment
Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords: realist evaluation, health systems research, mechanisms, context, intervention, theory of change, theory-based evaluation, methodological recommendations, conceptual clarity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks