Cited by 476Evaluation and Performance AssessmentHealth Policy Implementation ScienceAdvanced Causal Inference Techniques

The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.

Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research

Marchal, Bruno, van Belle, Sara, van Olmen, Josefien, Hoerée, Tom, Kegels, Guy·2012·Evaluation

DOI 10.1177/1356389012442444

Open this paper in CoChat — free account

Abstract

This overview aims to stimulate conceptual and practical discussions to help unlock the full potential of realist evaluation in health systems research. Based on a structured literature search, this review maps how the concepts of realist evaluation are applied in health systems research and which methodological problems are encountered. We found a great diversity in the depth of application of the philosophical concepts, use of terminology and scope of application in the research process. Terms of theory-driven evaluation, theories of change and realist evaluation are often used interchangeably. Diverging views exist regarding the nature of ‘mechanism’ and the difference between mechanism and essential context condition. A lack of methodological guidance was highlighted by many authors. Realist evaluation is slowly gaining traction in health systems research, but more clarity is needed concerning the definitions of mechanisms and context and how the configuration of context, mechanism and intervention can be described and assessed.

Integrity signals

Registry

Crossref · 10.1177/1356389012442444

Cites

93 of 93 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Social Theory and Social Structure
Ross, E. J., & Merton, R. Κ. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,318
Social Theory and Social Structure.
Bock, K. E., & Merton, R. Κ. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,687
Systems Thinking, Systems Practice
Checkland, P. B. (1981) · Cumulative Index of Computer Aided Architectural Design · Cited by 6,256
Systems thinking, systems practice
(1982) · European Journal of Operational Research · doi:10.1016/0377-2217(82)90206-5 · Cited by 3,579
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190
A set-theoretic approach to organizational configurations
Fiss, P. C. (2007) · Academy of Management Review · doi:10.5465/amr.2007.26586092 · Cited by 2,225
Evidence-based Policy
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
Critical realism: essential readings
(1999) · Choice Reviews Online · doi:10.5860/choice.36-5017 · Cited by 1,683
Designing and evaluating complex interventions to improve health care
Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L. (2007) · BMJ · doi:10.1136/bmj.39108.379965.be · Cited by 1,331
Theory-driven evaluations
Chen, H.-T. (1990) · Medical Entomology and Zoology · Cited by 1,227

Show all 92 references

Research works printed but not linked: 4

Connell JP, 1995, New Approaches To Evaluating Community Initiatives. Concepts, Methods, and Contexts
Kernick D, 2005, British Journal of Medical Practice, 55, 908
Pawson R, 1997, Realistic Evaluation
Weiss C, 1995, New Approaches to Evaluating Comprehensive Community Initiatives: Concepts, Methods and Contexts

Sorted by citations

What’s in a mechanism? Development of a key concept in realist evaluation
Dalkin, S. M., Greenhalgh, J., Jones, D., Cunningham, B., & Lhussier, M. (2015) · Implementation Science · doi:10.1186/s13012-015-0237-x · Cited by 896
RAMESES II reporting standards for realist evaluations
Wong, G., Westhorp, G., Manzano, A., Greenhalgh, J., Jagosh, J., & Greenhalgh, T. (2016) · BMC Medicine · doi:10.1186/s12916-016-0643-1 · Cited by 880
“Further, as noted in Item 5 above, the evaluation question(s), scope and design may evolve over the course of the evaluation [19].”
BackgroundInfluential
The craft of interviewing in realist evaluation
Manzano, A. (2016) · Evaluation · doi:10.1177/1356389016638615 · Cited by 691
Using theory of change to design and evaluate public health interventions: a systematic review
Breuer, E., Lee, L., De Silva, M., & Lund, C. (2015) · Implementation Science · doi:10.1186/s13012-016-0422-6 · Cited by 425
“Despite some fundamental differences in their theoretical underpinnings, many of these approaches are used interchangeably or together [18, 20].”
BackgroundInfluential
Realist Synthesis for Public Health: Building an Ontologically Deep Understanding of How Programs Work, For Whom, and In Which Contexts
Jagosh, J. (2019) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044451 · Cited by 346
What works for whom in which circumstances? On the need to move beyond the ‘what works?’ question in organizational intervention research
Nielsen, K. M., & Miraglia, M. (2016) · Human Relations · doi:10.1177/0018726716670226 · Cited by 385
“Challenges have been identified with regards to the definition and operationalization of mechanisms and context and the empirical testing of CMO-configurations, together with the confusion that some view specific elements as mechanisms while others define them as context and vice versa (Lacouture et al., 2015; Marchal…”
Influential
Data Analysis and Synthesis Within a Realist Evaluation: Toward More Transparent Methodological Approaches
Gilmore, B., McAuliffe, E., Power, J., & Vallières, F. (2019) · International Journal of Qualitative Methods · doi:10.1177/1609406919859754 · Cited by 296
“Consequently, and as highlighted by several authors (Byng, 2005; Marchal et al., 2012; Salter & Kothari, 2014; Westhorp, 2008), there is little guidance on specific analysis approaches and steps to use within a realist evaluation.”
Background
ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study
Wilson, P., Mathie, E., Keenan, J., McNeilly, E., Goodman, C., Howe, A., Poland, F., Staniszewska, S., Kendall, S., Munday, D., et al. (2015) · Health Services and Delivery Research · doi:10.3310/hsdr03380 · Cited by 297
“Realist evaluation (RE) is one of several theory-based approaches to evaluation developed within the social sciences, which seeks to address dynamism and context, rather than control for them, to explain more adequately how and, importantly, under what circumstances programmes, interventions or innovations will ‘work’…”
The concept of mechanism from a realist approach: a scoping review to facilitate its operationalization in public health program evaluation
Lacouture, A., Breton, É., Guichard, A., & Ridde, V. (2015) · Implementation Science · doi:10.1186/s13012-015-0345-7 · Cited by 295
“Moreover, two systematic reviews have highlighted all the ambiguity of the concept of mechanism in its comprehension and in its use for evaluating public health interventions [16, 17].”
Understanding ‘context’ in realist evaluation and synthesis
Greenhalgh, J., & Manzano, A. (2021) · International Journal of Social Research Methodology · doi:10.1080/13645579.2021.1918484 · Cited by 324
“Context as a collection of single timeless components Some papers listed catalogues of contexts and catalogues of mechanisms, rather than configurations of causal patterns, as noted previously by several scholars (Marchal et al., 2012; Pawson & ManzanoSantaella, 2012; Wong et al., 2016).”
Background
Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review
Salter, K. L., & Kothari, A. R. (2014) · Implementation Science · doi:10.1186/s13012-014-0115-y · Cited by 308
“Citations reported in the review of RE application in health system research conducted by Marchal and colleagues [13] were examined for possible inclusion in the present review.”
BackgroundInfluential
Achieving successful community engagement: a rapid realist review
de Weger, E., Vooren, N. V., Luijkx, K. G., Baan, C. A., & Drewes, H. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3090-1 · Cited by 232
“While most realist papers clearly highlight that CMO configurations were key to the development of the theories under discussion, most do not actually describe how the CMO configurations then led to those theories [47].”
Background
Building Realist Program Theory for Large Complex and Messy Interventions
Shearn, K., Allmark, P., Piercy, H., & Hirst, J. (2017) · International Journal of Qualitative Methods · doi:10.1177/1609406917741796 · Cited by 215
Protocol—the RAMESES II study: developing guidance and reporting standards for realist evaluation
Greenhalgh, T., Wong, G., Jagosh, J., Greenhalgh, J., Manzano, A., Westhorp, G., & Pawson, R. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008567 · Cited by 193
Quality and reporting standards, resources, training materials and information for realist evaluation: the RAMESES II project
Wong, G., Westhorp, G., Greenhalgh, J., Manzano, A., Jagosh, J., & Greenhalgh, T. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05280 · Cited by 196
“Two important prior publications have systematically shown that many so-called ‘realist evaluations’ were not applying the concepts appropriately and were, as a result, producing potentially misleading findings and recommendations.(12,13)”
BackgroundMethodInfluential
Challenges, solutions and future directions in the evaluation of service innovations in health care and public health
Raine, R., Fitzpatrick, R. M., Barratt, H., Bevan, G., Black, N. A., Boaden, R., Bower, P., Campbell, M., Denis, J., Devers, K. J., et al. (2016) · Health Services and Delivery Research · doi:10.3310/hsdr04160 · Cited by 197
Optimising the benefits of community health workers’ unique position between communities and the health sector: A comparative analysis of factors shaping relationships in four countries
Kok, M. C., Ormel, H., Broerse, J. E. W., Kane, S. S., Namakhoma, I., Otiso, L., Sidat, M., Kea, A. Z., Taegtmeyer, M., Theobald, S., et al. (2016) · Global Public Health · doi:10.1080/17441692.2016.1174722 · Cited by 196
People-centred science: strengthening the practice of health policy and systems research
Sheikh, K., George, A. S., & Gilson, L. (2014) · Health Research Policy and Systems · doi:10.1186/1478-4505-12-19 · Cited by 199
“Such knowledge includes capturing and explicating the complexity and interconnectedness of the various elements [36] in a way that has meaning to the people who make up the system and who can bring about appropriate systemic changes in different social contexts [37].”
Background
A realist evaluation of social prescribing: an exploration into the context and mechanisms underpinning a pathway linking primary care with the voluntary sector
Bertotti, M., Frostick, C., Hutt, P., Sohanpal, R., & Carnes, D. (2017) · Primary Health Care Research & Development · doi:10.1017/s1463423617000706 · Cited by 185
“It has been mostly used in the evaluation of health systems (Marchal et al., 2012), illicit drug deterrence programmes (Leone, 2008), shared care in mental health (Byng et al., 2008), community-based Primary Health Care Research & Development 2018; 19: 232–245 participatory research (Jagosh et al.,…”
Method
Travelling models and the challenge of pragmatic contexts and practical norms: the case of maternal health
Olivier de Sardan, J.-P., Diarra, A., & Moha, M. (2017) · Health Research Policy and Systems · doi:10.1186/s12961-017-0213-9 · Cited by 167
“years, however, various theoretical currents have emphasised the importance of context in health interventions, be it realist approaches to evaluation [15, 16], approaches”
Background
Evaluating community engagement in global health research: the need for metrics
the Ethics Working Group of the HIV Prevention Trials Network, MacQueen, K. M., Bhan, A., Frohlich, J., Holzer, J., & Sugarman, J. (2015) · BMC Medical Ethics · doi:10.1186/s12910-015-0033-9 · Cited by 157
“Realist evaluation presumes repeated iterations between theory development and empirical investigation at the micro level [52].”
Background
Elaborating the Context-Mechanism-Outcome configuration (CMOc) in realist evaluation: A critical realist perspective
de Souza, D. (2013) · Evaluation · doi:10.1177/1356389013485194 · Cited by 155
Revitalizing the setting approach – supersettings for sustainable impact in community health promotion
Bloch, P., Toft, U., Reinbach, H. C., Clausen, L. T., Mikkelsen, B. E., Poulsen, K., & Jensen, B. B. (2014) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-014-0118-8 · Cited by 157
“questioned whether theory-driven evaluation lives up to its promise [47,48], we have found it very useful to apply”
Background
A Mobile App to Improve Self-Management of Individuals With Type 2 Diabetes: Qualitative Realist Evaluation
Desveaux, L., Shaw, J., Saragosa, M., Soobiah, C., Marani, H., Hensel, J. M., Agarwal, P., Onabajo, N., Bhatia, R. S., & Jeffs, L. P. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8712 · Cited by 141
“Realist evaluation is a methodology used to unpack the black box of implementation [36] by purposively examining the actions required by those involved in an intervention to ensure its success, including participants and those involved in implementation.”
Method
What’s in a Realist Configuration? Deciding Which Causal Configurations to Use, How, and Why
de Weger, E., Vooren, N. V., Wong, G., Dalkin, S. M., Marchal, B., Drewes, H., & Baan, C. A. (2020) · International Journal of Qualitative Methods · doi:10.1177/1609406920938577 · Cited by 134
Self-citation
Using narratives to impact health policy-making: a systematic review
Fadlallah, R., El‐Jardali, F., Nomier, M. A., Hemadi, N., Arif, K., Langlois, É. V., & Akl, E. A. (2019) · Health Research Policy and Systems · doi:10.1186/s12961-019-0423-4 · Cited by 131
Realist review
Hunter, R. L., Gorely, T., Beattie, M., & Harris, K. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1969674 · Cited by 123
“The aim of a realist review is to develop plausible explanations from patterns of observations that have surfaced from the literature and create transferable theories that inform programme implementation in different settings (Marchal et al., 2012).”
Background
Understanding how and why audits work in improving the quality of hospital care: A systematic realist review
Hut-Mossel, L., Ahaus, K., Welker, G. A., & Gans, R. O. B. (2021) · PLoS ONE · doi:10.1371/journal.pone.0248677 · Cited by 142
Interventions to mitigate the effects of poverty and inequality on mental health
Wahlbeck, K., Cresswell‐Smith, J., Haaramo, P., & Parkkonen, J. (2017) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-017-1370-4 · Cited by 132
“Hence, we also included realist evaluations, which extend the analyses to include effects of social and historical context [26].”
Mechanisms, contexts and points of contention: operationalizing realist-informed research for complex health interventions
Shaw, J., Gray, C. S., Baker, G. R., Denis, J., Breton, M., Gutberg, J., Embuldeniya, G., Carswell, P., Dunham, A., McKillop, A., et al. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0641-4 · Cited by 111
“analysis beyond the individual’s reasoning [1, 3, 6, 11].”
Background
Qualitative research in nursing and health professions regulation
Squires, A., & Dorsen, C. (2018) · Journal of Nursing Regulation · doi:10.1016/s2155-8256(18)30150-9 · Cited by 108
Learning from success—Toward evidence-informed sustainability transitions in communities
Forrest, N., & Wiek, A. (2014) · Environmental Innovation and Societal Transitions · doi:10.1016/j.eist.2014.01.003 · Cited by 104
Examining the Application of Retroductive Theorizing in Realist-Informed Studies
Mukumbang, F. C., Kabongo, E. M., & Eastwood, J. G. (2021) · International Journal of Qualitative Methods · doi:10.1177/16094069211053516 · Cited by 102
“Marchal et al. (2012) explored if realist evaluation studies remained faithful to realist principles and found important gaps that needed to be addressed regarding the reporting of such studies. In response to their findings, while following the tradition of developing guidelines for reporting different types of studi…”
Background
Towards an integrated framework for evaluating transformative innovation policy
Haddad, C. R., & Bergek, A. (2022) · Research Policy · doi:10.1016/j.respol.2022.104676 · Cited by 99
Unearthing how, why, for whom and under what health system conditions the antiretroviral treatment adherence club intervention in South Africa works: A realist theory refining approach
Mukumbang, F. C., Marchal, B., Belle, S. V., & van Wyk, B. E. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3150-6 · Cited by 90
“testable hypotheses of the programme, intervention or policy and testing these hypotheses in identified cases leads to case-specific theories that provide propositions that can be tested and refined [39].”
BackgroundSelf-citation
Anchoring contextual analysis in health policy and systems research: A narrative review of contextual factors influencing health committees in low and middle income countries
George, A. S., Scott, K., Garimella, S., Mondal, S., Ved, R. R., & Sheikh, K. (2015) · Social Science & Medicine · doi:10.1016/j.socscimed.2015.03.049 · Cited by 93
“Rather than rely on empty arenas, generic lists or amorphous detail, our approach to understanding context follows realist perspectives of evaluation that emphasize the development of program theories to guide exploration (Pawson and Tilley, 1997; Marchal et al., 2012).”
Background
What Is This Thing Called a Mechanism? Findings From a Review of Realist Evaluations
Lemire, S. T., Kwako, A., Nielsen, S. B., Christie, C. A., Donaldson, S. I., & Leeuw, F. L. (2020) · New Directions for Evaluation · doi:10.1002/ev.20428 · Cited by 88
Optimal NHS service delivery to care homes: a realist evaluation of the features and mechanisms that support effective working for the continuing care of older people in residential settings
Goodman, C. M., Davies, S. L., Gordon, A. L., Dening, T., Gage, H., Meyer, J. E., Schneider, J. M., Bell, B., Jordan, J., Martin, F. C., et al. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05290 · Cited by 87
“process is challenging.(160) Over the course of the project we refined our understanding of how processes”
Background
Collaboration between parents and SLTs produces optimal outcomes for children attending speech and language therapy: Gathering the evidence
Klatte, I. S., Lyons, R., Davies, K., Harding, S. A., Marshall, J., McKean, C., & Roulstone, S. (2020) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12538 · Cited by 85
Evaluating a major innovation in hospital design: workforce implications and impact on patient and staff experiences of all single room hospital accommodation
Maben, J., Griffiths, P., Penfold, C., Simon, M., Pizzo, E., Anderson, J. E., Robert, G., Hughes, J., Murrells, T., Brearley, S. A., et al. (2015) · Health Services and Delivery Research · doi:10.3310/hsdr03030 · Cited by 84
Explaining the impact of mHealth on maternal and child health care in low- and middle-income countries: a realist synthesis
Kabongo, E. M., Mukumbang, F. C., Delobelle, P., & Nicol, E. (2021) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-021-03684-x · Cited by 93
Advancing the application of systems thinking in health: a realist evaluation of a capacity building programme for district managers in Tumkur, India
Prashanth, N. S., Marchal, B., Devadasan, N., Kegels, G., & Criel, B. (2014) · Health Research Policy and Systems · doi:10.1186/1478-4505-12-42 · Cited by 81
“Its potential as an evaluation approach for complex health systems problems has gained interest over the last decade [24-28].”
BackgroundSelf-citation
Can “realist” randomised controlled trials be genuinely realist?
Belle, S. V., Wong, G., Westhorp, G., Pearson, M., Emmel, N. D., Manzano, A., & Marchal, B. (2016) · Trials · doi:10.1186/s13063-016-1407-0 · Cited by 77
Self-citation
A realist synthesis of the effect of social accountability interventions on health service providers’ and policymakers’ responsiveness
Lodenstein, E., Dieleman, M. A., Gerretsen, B., & Broerse, J. E. W. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-98 · Cited by 83
“Context Actors or factors that are external to the intervention, occurring independently of the outcome or influencing the outcome [32].”
Background
From favours to entitlements: community voice and action and health service quality in Zambia
Schaaf, M., Topp, S. M., & Ngulube, M. (2017) · Health Policy and Planning · doi:10.1093/heapol/czx024 · Cited by 76
Communicable disease control programmes and health systems: an analytical approach to sustainability
Shigayeva, A., & Coker, R. J. (2015) · Health Policy and Planning · doi:10.1093/heapol/czu005 · Cited by 63
A realist approach to eliciting the initial programme theory of the antiretroviral treatment adherence club intervention in the Western Cape Province, South Africa
Mukumbang, F. C., Marchal, B., Belle, S. V., & van Wyk, B. E. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0503-0 · Cited by 70
“It was found by Marchal and colleagues [58] that in attempting to specify the context-mechanism-outcome configurations while formulating realist programme theories, some authors fail to demonstrate the explanatory nature of”
BackgroundSelf-citation
Critical realism in nursing: an emerging approach.
Schiller, C. J. (2016) · Nursing Philosophy · doi:10.1111/nup.12107 · Cited by 84
What Are the Role and Impact of Public-Private Partnerships in Education? A Realist Evaluation of the Chilean Education Quasi-Market
Verger, A., Bonal, X., & Zancajo, A. (2016) · Comparative Education Review · doi:10.1086/685557 · Cited by 70
“Of course the realist evaluation approach has itself been the subject of criticism and is not without limitations (see Marchal et al. 2012; Porter 2015).”
Background
Evaluating the effectiveness of a multifaceted, multilevel continuous quality improvement program in primary health care: developing a realist theory of change
Schierhout, G., Hains, J., Si, D., Kennedy, C., Cox, R., Kwedza, R., O’Donoghue, L., Fittock, M., Brands, J., Lonergan, K., et al. (2013) · Implementation Science · doi:10.1186/1748-5908-8-119 · Cited by 67
“Our research strategy of developing middle-range theory based on program implementers’ views is consistent with recommendations from a recent overview of the use of realist evaluation methods in health systems research, which proposes this as an explicit additional step in situations where there is little to inform a-…”
Background
External Validity and Policy Adaptation: From Impact Evaluation to Policy Design
Williams, M. J. (2018) · The World Bank Research Observer · doi:10.1093/wbro/lky010 · Cited by 66
How to phrase critical realist interview questions in applied social science research
Brönnimann, A. (2021) · Journal of Critical Realism · doi:10.1080/14767430.2021.1966719 · Cited by 58
“However, realist evaluators face problems when left without an interview starting point, because the phenomena of interest can only be explained very little or not at all by existing theory (Marchal et al. 2012).”
Background
Towards a programme theory for fidelity in the evaluation of complex interventions
Masterson‐Algar, P., Burton, C. R., Rycroft‐Malone, J., Sackley, C. M., & Walker, M. F. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12174 · Cited by 67
Evaluation of an organizational health intervention for low-skilled workers and immigrants
Busch, C., Koch, T., Clasen, J., Winkler, E., & Vowinkel, J. (2017) · Human Relations · doi:10.1177/0018726716682308 · Cited by 66
“‘Change occurs when interventions, combined with the right contextual factors, release the generative mechanisms’ (Marchal et al., 2012: 202).”
Background
Combining Theories of Change and Realist Evaluation in practice: Lessons from a research on evaluation study
Rolfe, S. (2019) · Evaluation · doi:10.1177/1356389019835229 · Cited by 68
“Thirdly, some studies have struggled to identify appropriate theories, either because practitioners are unclear about their approach (as with ToC), or because there is insufficient or conflicting academic theory (Adams et al., 2016; Marchal et al., 2012).”
BackgroundInfluential
Person‐centred care in primary care: What works for whom, how and in what circumstances?
Ahmed, A., van den Muijsenbergh, M. E. T. C., & Vrijhoef, H. J. (2022) · Health & Social Care in the Community · doi:10.1111/hsc.13913 · Cited by 66
“…inherent to realist research, is the lack of conceptual clarity of the constructs context (J. Greenhalgh & Manzano, 2021; Renmans et al., 2020) and mechanisms (Lacouture et al., 2015; Lemire et al., 2019), which makes assigning items to the different constructs difficult (Marchal et al., 2012).”
Background
Towards an agenda for implementation science in global health: there is nothing more practical than good (social science) theories
Belle, S. V., van de Pas, R., & Marchal, B. (2017) · BMJ Global Health · doi:10.1136/bmjgh-2016-000181 · Cited by 65
Self-citation
Optimal healthcare delivery to care homes in the UK: a realist evaluation of what supports effective working to improve healthcare outcomes
Gordon, A. L., Goodman, C. M., Davies, S. L., Dening, T., Gage, H., Meyer, J. E., Schneider, J. M., Bell, B., Jordan, J., Martin, F. C., et al. (2017) · Age and Ageing · doi:10.1093/ageing/afx195 · Cited by 65
“For example, mapping of service provision and networks of support around the care homes suggested additional contexts that led, or did not lead, to a sense of common purpose [13].”
Background
Making implementation science more real
Sarkies, M., Francis‐Auton, E., Long, J. C., Pomare, C., Hardwick, R., & Braithwaite, J. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01661-2 · Cited by 66
The uncritical realism of realist evaluation
Porter, S. (2015) · Evaluation · doi:10.1177/1356389014566134 · Cited by 67
“…and action are analytically separate, in combination with the deconstruction of context into the mechanisms that constitute it, will clear up much of the methodological confusion that has dogged evaluators attempting to operationalise the CMO formula in their work (vide Marchal et al., 2012).”
Background
Unpacking context in realist evaluations: Findings from a comprehensive review
Nielsen, S. B., Lemire, S. T., & Tangsig, S. (2021) · Evaluation · doi:10.1177/13563890211053032 · Cited by 63
“The fifth review by Manzano (2016: 344) describes the sample of studies as the studies included in Marchal et al. (2012) plus “Three more years of published realist evaluations in health.””
Background
Supporting general hospital staff to provide dementia sensitive care: A realist evaluation
Handley, M., Bunn, F., & Goodman, C. M. (2018) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2018.10.004 · Cited by 62
Implications of the Changing Conversation About Causality for Evaluators
Gates, E., & Dyson, L. (2016) · American Journal of Evaluation · doi:10.1177/1098214016644068 · Cited by 64
Methods in realist evaluation: A mapping review
Renmans, D., & Castellano Pleguezuelo, V. (2023) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2022.102209 · Cited by 68
Bringing critical realism to nursing practice: Roy Bhaskar's contribution
Williams, L., Rycroft‐Malone, J., & Burton, C. R. (2016) · Nursing Philosophy · doi:10.1111/nup.12130 · Cited by 78
‘The government cannot do it all alone’: realist analysis of the minutes of community health committee meetings in Nigeria
Abimbola, S., Molemodile, S. K., Okonkwo, O. A., Negin, J., Jan, S., & Martiniuk, A. (2015) · Health Policy and Planning · doi:10.1093/heapol/czv066 · Cited by 58
“Evaluating complex interventions is methodologically challenging and theory-driven analyses have been demonstrated as being well suited for understanding the workings of complex health system interventions (Campbell et al. 2007; Marchal et al. 2012).”
Integration of robotic surgery into routine practice and impacts on communication, collaboration, and decision making: a realist process evaluation protocol
Randell, R., Greenhalgh, J., Hindmarsh, J., Dowding, D., Jayne, D. G., Pearman, A. D., Gardner, P. H., Croft, J., & Kotzé, A. (2014) · Implementation Science · doi:10.1186/1748-5908-9-52 · Cited by 60
“Realist evaluation is increasingly popular as a method for evaluating the implementation of complex interventions in healthcare [32-36] and has been applied in a variety of fields within health systems research [37].”
Method
Realist evaluation of the antiretroviral treatment adherence club programme in selected primary healthcare facilities in the metropolitan area of Western Cape Province, South Africa: a study protocol
Mukumbang, F. C., Belle, S. V., Marchal, B., & van Wyk, B. E. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-009977 · Cited by 59
“50 Therefore, realist evaluation can provide a sound framework by which the context and mechanisms and how they in ﬂ uence the outcome of an intervention could be studied.”
BackgroundInfluentialSelf-citation
L’approche réaliste pour l’évaluation de programmes et la revue systématique
Robert, É., & Ridde, V. (2014) · Mesure et évaluation en éducation · doi:10.7202/1025741ar · Cited by 83
“(p. 52) Marchal et al. (2012) le rappellent également : « définir le ‘contexte’ et séparer le ‘mécanisme’ du ‘contexte’ reste une tâche difficile».”
Influential
Cost and impact of policies to remove and reduce fees for obstetric care in Benin, Burkina Faso, Mali and Morocco
On behalf of the FEMHealth team, Witter, S., Boukhalfa, C., Cresswell, J. A., Daou, Z., Filippi, V., Ganaba, R., Goufodji, S. B., Lange, I. L., Marchal, B., et al. (2016) · International Journal for Equity in Health · doi:10.1186/s12939-016-0412-y · Cited by 58
Self-citation
Labour inspections and the prevention of psychosocial risks at work: A realist synthesis
Weissbrodt, R., & Giauque, D. (2017) · Safety Science · doi:10.1016/j.ssci.2017.02.012 · Cited by 64
“It is indeed often difficult to distinguish between contexts and mechanisms (Dalkin et al., 2015) or between mechanisms and interventions (Marchal et al., 2012).”
Background
An overview of realist evaluation for simulation-based education
Graham, A., & McAleer, S. (2018) · Advances in Simulation · doi:10.1186/s41077-018-0073-6 · Cited by 57
“This interplay between individuals and the educational programme means that the causal mechanisms are located in the social relations and context as well as the individuals [28].”
Background
Putting context centre stage: evidence from a systems evaluation of an area based empowerment initiative in England
Orton, L., Halliday, E. C., Collins, M., Egan, M., Lewis, S., Ponsford, R., Powell, K., Salway, S., Townsend, A. F., Whitehead, M. M., et al. (2016) · Critical Public Health · doi:10.1080/09581596.2016.1250868 · Cited by 56
What works in ‘real life’ to facilitate home deaths and fewer hospital admissions for those at end of life?: results from a realist evaluation of new palliative care services in two English counties
Wye, L., Lasseter, G., Percival, J., Duncan, L., Simmonds, B., & Purdy, S. (2014) · BMC Palliative Care · doi:10.1186/1472-684x-13-37 · Cited by 55
“But few applied examples of realist evaluation exist, as a recent review found only 20 published studies to date [8].”
Background
‘Same, but different’: A mixed-methods realist evaluation of a cluster-randomized controlled participatory organizational intervention
Abildgaard, J. S., Nielsen, K. M., Wåhlin-Jacobsen, C. D., Maltesen, T., Christensen, K. B., & Holtermann, A. (2019) · Human Relations · doi:10.1177/0018726719866896 · Cited by 54
“Mechanisms are not only individual factors (such as change readiness) but also collective factors related to managerial behaviour and culture (Marchal et al., 2012).”
Background
A realist evaluation of student use of a virtual reality smartphone application in undergraduate legal education
McFaul, H., & FitzGerald, E. (2019) · British Journal of Educational Technology · doi:10.1111/bjet.12850 · Cited by 59
“rarely been applied to the evaluation of educational technology (King et al., 2016; Marchal et al., 2012) and, as an emergent theoretical framework, a settled consensus on its practical application has yet to emerge (Marchal et al.”
BackgroundMethodInfluential
First Things First: How to Elicit the Initial Program Theory for a Realist Evaluation of Complex Integrated Care Programs
SMEETS, R. G. M., HERTROIJS, D. F. L., MUKUMBANG, F. C., KROESE, M. E. A. L., RUWAARD, D., & ELISSEN, A. M. J. (2022) · The Milbank Quarterly · doi:10.1111/1468-0009.12543 · Cited by 53
Understanding the motivation and performance of community health volunteers involved in the delivery of health programmes in Kampala, Uganda: a realist evaluation
Vareilles, G., Marchal, B., Kane, S. S., Petrič, T., Pictet, G., & Pommier, J. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008614 · Cited by 53
Self-citation
How Can Autonomy Be Maintained and Informal Care Improved for People With Dementia Living in Residential Care Facilities: A Systematic Literature Review
Boumans, J., van Boekel, L. C., Baan, C. A., & Luijkx, K. G. (2018) · The Gerontologist · doi:10.1093/geront/gny096 · Cited by 51
Realist evaluations in low- and middle-income countries: reflections and recommendations from the experiences of a foreign researcher
Gilmore, B. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2019-001638 · Cited by 55
The contributions of scientific realism and critical realism to realist evaluation
Mukumbang, F. C., de Souza, D., & Eastwood, J. G. (2023) · Journal of Critical Realism · doi:10.1080/14767430.2023.2217052 · Cited by 71
“Please see Correction (http://dx.doi.org/10.1080/14767430.2023.2232661) methodological foundations, especially among novice realist researchers, have been noted (Marchal et al. 2012).”
BackgroundInfluential
The effect of leadership on public service motivation: a multiple embedded case study in Morocco
Belrhiti, Z., Damme, W. V., Belalia, A., & Marchal, B. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-033010 · Cited by 59
Self-citation
A realist process evaluation of robot-assisted surgery: integration into routine practice and impacts on communication, collaboration and decision-making
Randell, R., Honey, S., Hindmarsh, J., Alvarado, N., Greenhalgh, J., Pearman, A. D., Long, A., Cope, A., Gill, A., Gardner, P. H., et al. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05200 · Cited by 48
“Consequently, realist evaluation has been used for studying the implementation of a number of complex interventions in health care.(47) In realist evaluation, interventions in and of themselves are not seen as determining outcomes.”
Method
Action leveraging evidence to reduce perinatal mortality and morbidity (ALERT): study protocol for a stepped-wedge cluster-randomised trial in Benin, Malawi, Tanzania and Uganda
Akuze, J., Annerstedt, K. S., Beňová, L., Chipeta, E. K., Dossou, J., Groß, M. M., Kidanto, H. L., Marchal, B., Alvesson, H. M., Pembe, A. B., et al. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-07155-z · Cited by 44
“4 The realist research cycle [59] Akuze et al.”
BackgroundSelf-citation
Challenges related to the analytical process in realist evaluation and latest developments on the use of NVivo from a realist perspective
Bergeron, D. A., & Gaboury, I. (2019) · International Journal of Social Research Methodology · doi:10.1080/13645579.2019.1697167 · Cited by 47
“The process of developing of a C-M-O configuration ultimately allows refinement of an initial program theory (Marchal et al., 2012; Ridde et al., 2012) into a more abstract theory, that can facilitate generalization to other contexts (Blaise et al., 2010; Dickinson, 2006; Greenhalgh et al., 2017d;…”
BackgroundMethodInfluential
Evaluation of Capacity-Building Program of District Health Managers in India: A Contextualized Theoretical Framework
Srinivas, P. N., Marchal, B., Macq, J., Kegels, G., & Criel, B. (2014) · Frontiers in Public Health · doi:10.3389/fpubh.2014.00089 · Cited by 48
Self-citation
Realist review protocol for understanding the real-world barriers and enablers to practitioners implementing self-management support to people living with and beyond cancer
Kantilal, K., Hardeman, W., Whiteside, H., Karapanagiotou, E. M., Small, M., & Bhattacharya, D. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-037636 · Cited by 43
Towards a framework for enhancing the performance of NHS boards: a synthesis of the evidence about board governance, board effectiveness and board development
Chambers, N., Harvey, G., Mannion, R., Bond, J., & Marshall, J. (2013) · Health Services and Delivery Research · doi:10.3310/hsdr01060 · Cited by 47
Contexts, Mechanisms, and Outcomes That Matter in Dutch Community-Based Physical Activity Programs Targeting Socially Vulnerable Groups
Herens, M. C., Wagemakers, A., Vaandrager, L., van Ophem, J., & Koelen, M. A. (2016) · Evaluation & the Health Professions · doi:10.1177/0163278716652940 · Cited by 45
“The analysis of qualitative data from interview transcripts is then based on coding in terms of outcomes as observed by respondents, context conditions, and description of underlying mechanisms in the actual intervention (Marchal et al., 2012).”
BackgroundMethod
Exploring the use of Soft Systems Methodology with realist approaches: A novel way to map programme complexity and develop and refine programme theory
Dalkin, S. M., Lhussier, M., Williams, L., Burton, C. R., & Rycroft‐Malone, J. (2018) · Evaluation · doi:10.1177/1356389017749036 · Cited by 55
“…around defining context and mechanisms has raised dilemmas for the evaluator (Pedersen et al., 2012, Astbury and Leeuw, 2010, Jagosh et al., 2013, Marchal et al., 2012) although recent methodological guidelines and publications have come some way in overcoming this (Wong et al., 2016, Wong et…”
Implementing large-system, value-based healthcare initiatives: a realist study protocol for seven natural experiments
Sarkies, M., Francis‐Auton, E., Long, J. C., Partington, A., Pomare, C., Nguyen, H. M., Wu, W., Westbrook, J. I., Day, R. O., Lévesque, J., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-044049 · Cited by 44
Methodological reflections on using qualitative research to explore the causal mechanisms of complex health interventions
Bonell, C., Warren, E., & Meléndez-Torres, G. (2022) · Evaluation · doi:10.1177/13563890221086309 · Cited by 40
“Some realist evaluators are open to the use of quantitative alongside qualitative data to examine mechanisms (Marchal et al., 2012).”
BackgroundInfluential
Realist evaluation: an immanent critique
Porter, S. (2015) · Nursing Philosophy · doi:10.1111/nup.12100 · Cited by 44
“The momentum started around 2009 (Marchal et al., 2012), and since then, the flow of publications has increased dramatically (see e.”
Background
What Works for whom in which Circumstances? An Integrated Realist Evaluation Model for Organisational Interventions
Roodbari, H., Nielsen, K. M., & Axtell, C. M. (2023) · Scandinavian Journal of Work and Organizational Psychology · doi:10.16993/sjwop.171 · Cited by 42
Mechanisms for achieving adolescent-friendly services in Ecuador: a realist evaluation approach
Goicolea, I., Coe, A., Hurtig, A., & Sebastiån, M. S. (2012) · Global Health Action · doi:10.3402/gha.v5i0.18748 · Cited by 42
Integrating palliative care within acute stroke services: developing a programme theory of patient and family needs, preferences and staff perspectives
Burton, C. R., & Payne, S. A. (2012) · BMC Palliative Care · doi:10.1186/1472-684x-11-22 · Cited by 41
“In keeping with realist approaches to theory building [25], the higher-level, abstract mechanisms are presented in this paper.”
Background
The SMART personalised self-management system for congestive heart failure: results of a realist evaluation
Bartlett, Y. K., Haywood, A., Bentley, C. L., Parker, J. C., Hawley, M., Mountain, G. A., & Mawson, S. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-014-0109-3 · Cited by 39
“[14,15]), but has recently been adapted for use with health related interventions (including health technology interventions e.”
BackgroundMethodInfluential
Protocol: Realist synthesis of the impact of unemployment insurance policies on poverty and health
Molnar, A., O'Campo, P. J., Ng, E., Mitchell, C., Muntaner, C., Renahy, É., John, A. S., & Shankardass, K. (2014) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2014.09.002 · Cited by 41
Enabling conditions for effective marine spatial planning
Zuercher, R., Ban, N. C., Flannery, W., Guerry, A. D., Halpern, B. S., Magris, R. A., Mahajan, S. L., Motzer, N., Spalding, A. K., Stelzenmüller, V., et al. (2022) · Marine Policy · doi:10.1016/j.marpol.2022.105141 · Cited by 38
“Theory-based evaluation [28,29,48,85,97] Various approaches to theory-based evaluation, including realist evaluation, provide avenues for operationalizing this framework.”
Method
Power dynamics and intersectoral collaboration for health in low- and middle-income countries: a realist review
Aivalli, P., Dada, S., Gilmore, B., Srinivas, P. N., & Brún, A. D. (2025) · Health Policy and Planning · doi:10.1093/heapol/czaf022 · Cited by 57
“Instead, ISC requires an approach that examines generative causation, focusing on how and why specific mechanisms—such as trust-building, resource negotiation, or bureaucratic discretion—produce outcomes under particular conditions (Marchal et al. 2012).”

Authors: Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G.
Affiliations: ITM-A, Belgium
BTC, Belgium
ORCID: Marchal, Bruno
Kegels, Guy
Venue: Evaluation
Publisher: SAGE Publications
Published: 2012-04
Volume: 18
Issue: 2
Pages: 192-212
ISSN: 1356-3890 (ISSN-L)
1461-7153
DOI: 10.1177/1356389012442444
OpenAlex: W2077589521
Cited by: 476
References: 92
Citations by year: 2026: 22 · 2025: 29 · 2024: 41 · 2023: 29 · 2022: 51 · 2021: 64 · 2020: 31 · 2019: 40 · 2018: 37 · 2017: 28 · 2016: 31 · 2015: 34 · 2014: 24 · 2013: 12 · 2012: 3
FWCI: 30.83
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Evaluation (publishedVersion)
Topics: Social Sciences › Public Administration › Evaluation and Performance Assessment
Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords: realist evaluation, health systems research, mechanisms, context, intervention, theory of change, theory-based evaluation, methodological recommendations, conceptual clarity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks