The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research
Abstract
This overview aims to stimulate conceptual and practical discussions to help unlock the full potential of realist evaluation in health systems research. Based on a structured literature search, this review maps how the concepts of realist evaluation are applied in health systems research and which methodological problems are encountered. We found a great diversity in the depth of application of the philosophical concepts, use of terminology and scope of application in the research process. Terms of theory-driven evaluation, theories of change and realist evaluation are often used interchangeably. Diverging views exist regarding the nature of ‘mechanism’ and the difference between mechanism and essential context condition. A lack of methodological guidance was highlighted by many authors. Realist evaluation is slowly gaining traction in health systems research, but more clarity is needed concerning the definitions of mechanisms and context and how the configuration of context, mechanism and intervention can be described and assessed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Marchal et al. [18], in a systematic review on realist evaluation in health systems research, also noted that ToC and realist evaluation were often used together or interchangeably.”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [18]
In the citing paper
“Both a published literature review 112 and critical discussions in the applied literature cite several methodological challenges associated with a RE approach, including deciding what constitutes a mechanism, 127 and indeed differentiating between what is a mechanism and what is context.”
From ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study (2015) · cites this paper as [112]
Citations
Cites
Social Theory and Social Structure
Ross, E. J., & Merton, R. Κ. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,318
Social Theory and Social Structure.
Bock, K. E., & Merton, R. Κ. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,687
Systems Thinking, Systems Practice
Checkland, P. B. (1981) · Cumulative Index of Computer Aided Architectural Design · Cited by 6,256
Systems thinking, systems practice
(1982) · European Journal of Operational Research · doi:10.1016/0377-2217(82)90206-5 · Cited by 3,579
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190
A set-theoretic approach to organizational configurations
Fiss, P. C. (2007) · Academy of Management Review · doi:10.5465/amr.2007.26586092 · Cited by 2,225
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
Critical realism: essential readings
(1999) · Choice Reviews Online · doi:10.5860/choice.36-5017 · Cited by 1,683
Designing and evaluating complex interventions to improve health care
Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L. (2007) · BMJ · doi:10.1136/bmj.39108.379965.be · Cited by 1,331
Chen, H.-T. (1990) · Medical Entomology and Zoology · Cited by 1,227
Research works printed but not linked: 4
- Connell JP, 1995, New Approaches To Evaluating Community Initiatives. Concepts, Methods, and Contexts
- Kernick D, 2005, British Journal of Medical Practice, 55, 908
- Pawson R, 1997, Realistic Evaluation
- Weiss C, 1995, New Approaches to Evaluating Comprehensive Community Initiatives: Concepts, Methods and Contexts
Cited by
Sorted by citations
Items 101–200 of 474
Belle, S. V., Rifkin, S. B., & Marchal, B. (2017) · BMC Health Services Research · doi:10.1186/s12913-017-2627-z · Cited by 38
Self-citationGoicolea, I., Hurtig, A., Sebastiån, M. S., Vives‐Cases, C., & Marchal, B. (2015) · BMC Health Services Research · doi:10.1186/s12913-015-0899-8 · Cited by 37
“The second stage of the qualitative data analysis consisted of searching for CMO configurations or patterns (demi-regularities) that provide an explanation for the observed outcomes [18, 19, 30].”
MethodSelf-citationAdams, A. M., Sedalia, S., McNab, S., & Sarker, M. (2015) · Health Policy and Planning · doi:10.1093/heapol/czv053 · Cited by 37
Villeval, M., Bidault, E., Shoveller, J., Alias, F., Basson, J.-C., Frasse, C., Génolini, J., Pons, E., Verbiguié, D., Grosclaude, P., et al. (2016) · International Journal of Public Health · doi:10.1007/s00038-016-0809-9 · Cited by 37
“However, these approaches may seem difficult to implement, notably due to a lack of methodological guidance (Marchal et al. 2012).”
Zulu, J. M., Goicolea, I., Kinsman, J., Sandøy, I. F., Blystad, A., Mulubwa, C., Makasa, M., Michelo, C., Musonda, P., & Hurtig, A. (2018) · Reproductive Health · doi:10.1186/s12978-018-0590-8 · Cited by 37
Nielsen, K. M., & Shepherd, R. (2022) · Work & Stress · doi:10.1080/02678373.2022.2028318 · Cited by 44
“The Blume et al. (2010) meta-analysis did not focus on the impact of training design and content, however, these influence the extent to which learning is achieved (e.g. Ford et al., 2018; Marchal et al., 2012).”
BackgroundHernández, A., Hurtig, A., Dahlblom, K., & Sebastiån, M. S. (2014) · BMC Health Services Research · doi:10.1186/1472-6963-14-112 · Cited by 35
“tems operates at the inter-personal level is limited [20].”
BackgroundAn integrative framework to evaluate impacts of complex tourism change initiatives
Roodbari, H., & Olya, H. (2023) · Tourism Management · doi:10.1016/j.tourman.2023.104829 · Cited by 41
Goicolea, I., Vives‐Cases, C., Hurtig, A., Marchal, B., Briones‐Vozmediano, E., Otero‐García, L., García‐Quinto, M., & Sebastiån, M. S. (2015) · PLoS ONE · doi:10.1371/journal.pone.0135167 · Cited by 35
“Realist evaluation is a type of theory-driven evaluation that aims to ascertain why, how and under what circumstances programmes succeed or fail [19–22].”
BackgroundSelf-citationFaculty development for educators: a realist evaluation
Sorinola, O. O., Thistlethwaite, J. E., Davies, D., & Peile, E. (2014) · Advances in Health Sciences Education · doi:10.1007/s10459-014-9534-4 · Cited by 34
“Realist evaluation is now considered as one of the ‘theory-driven inquiry’ schools (Marchal et al. 2012).”
BackgroundArchibald, M. M., Lawless, M., Harvey, G., & Kitson, A. A. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-021775 · Cited by 35
“A middle-range theory is generated, which lies between the working hypothesis and a fully operational, explanatory theory.(15) 16 Data”
BackgroundKerr, H., Price, J. E., Nicholl, H. M., & O’Halloran, P. D. (2018) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2018.06.015 · Cited by 34
“Context-mechanism-outcome configurations are a structured way of defining programme theories (Westhorp et al., 2011) by examining “how the context and mechanisms influence the outcomes of an intervention” (Marchal et al., 2012, p. 201).”
BackgroundNegotiating effectiveness in transnational advocacy evaluation
Arensman, B., & van Wessel, M. (2017) · Evaluation · doi:10.1177/1356389017733210 · Cited by 12
“Growing pressure for results has pushed NGOs and donors into evaluations based on traditions of prediction and control, seeking general patterns of cause–effect relations (Armytage, 2011; Riddell, 2014), based on the assumption that evaluation is value-free (Marchal et al., 2012).”
BackgroundA realist evaluation of an antenatal programme to change drinking behaviour of pregnant women
Doi, L., Jepson, R. G., & Cheyne, H. (2015) · Midwifery · doi:10.1016/j.midw.2015.06.007 · Cited by 33
Jeffries, M., Phipps, D. L., Howard, R. L., Avery, A. J., Rodgers, S., & Ashcroft, D. M. (2017) · BMC Health Services Research · doi:10.1186/s12913-017-2131-5 · Cited by 33
Punton, M., & Vogel, I. (2020) · New Directions for Evaluation · doi:10.1002/ev.20427 · Cited by 34
Vareilles, G., Pommier, J., Kane, S. S., Pictet, G., & Marchal, B. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-006752 · Cited by 34
Self-citationEvaluation in African contexts: The promises of participatory approaches in theory-based evaluations
Mbava, N. P., & Dahler‐Larsen, P. (2019) · African Evaluation Journal · doi:10.4102/aej.v7i1.383 · Cited by 33
“Theory-driven evaluation could potentially be resource intensive as it requires rigorous testing of programme theory so as to ascertain plausibility (Marchal et al. 2012).”
Jeffs, L. P., Jain, A. K., Man, R., Onabajo, N., Desveaux, L., Shaw, J., Hensel, J. M., Agarwal, P., Saragosa, M., Jamieson, T., et al. (2017) · BMC Nephrology · doi:10.1186/s12882-017-0557-y · Cited by 32
“This analysis will also yield a series of iteratively refined statements of the relationships between 1) key contextual factors, 2) the mechanisms by which they effect the implementation of the virtual care interventions, and 3) the impact on the outcomes of the intervention themselves referred to as “Context-Mechanis…”
BackgroundMethodInfluentialSport policy evaluation: what do we know and how might we move forward?
Chen, S. (2018) · International Journal of Sport Policy and Politics · doi:10.1080/19406940.2018.1488759 · Cited by 36
“Although realist evaluation has modest popularity in mainstream literature areas (Marchal et al. 2012), its application in the field of sport is still limited and young (Bell 2004, Hughes 2013, Chen and Henry 2016, 2017, Girginov 2016, Harris 2018).”
BackgroundLowe, H., Brown, L. J., Ahmad, A., Daruwalla, N., Gram, L., Osrin, D., Panchal, K., Watson, D., Zimmerman, C., & Mannell, J. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.115064 · Cited by 35
Riggs, E., Block, K., Warr, D. J., & Gibbs, L. F. (2013) · Health Promotion International · doi:10.1093/heapro/dat022 · Cited by 32
“(Pawson and Tilley, 1997; Marchal et al., 2012).”
BackgroundUsing non-participant observation to uncover mechanisms: Insights from a realist evaluation
Handley, M., Bunn, F., Lynch, J. K., & Goodman, C. M. (2019) · Evaluation · doi:10.1177/1356389019869036 · Cited by 31
“Researchers using realist evaluation have highlighted the difficulties of identifying mechanisms: such as interpreting them as a context or defining intervention activities as mechanisms (Greenhalgh et al., 2009; Marchal et al., 2012; Rycroft-Malone et al., 2010).”
BackgroundExploring taught masters education for healthcare practitioners: a systematic review of literature
Madi, M., Hamzeh, H., Griffiths, M., Rushton, A., & Heneghan, N. R. (2019) · BMC Medical Education · doi:10.1186/s12909-019-1768-7 · Cited by 34
“Although such contextual factors are arguably external to programme activities [17], they are integral to a programme’s learning culture by virtue of the learners’ biographies [18].”
BackgroundCruz-Martínez, R. R., Wentzel, J., Asbjørnsen, R. A., Noort, P. D., van Niekerk, J. M., Sanderman, R., & van Gemert‐Pijnen, J. E. W. C. (2020) · Journal of Medical Internet Research · doi:10.2196/16157 · Cited by 32
“These ingredients are instead derived from research and intervention-building frameworks from behavioral [74], medical [76-78], or sociological [83,84] areas”
BackgroundMethodClark, J. D., & Kearns, A. (2015) · Local Economy The Journal of the Local Economy Policy Unit · doi:10.1177/0269094215601993 · Cited by 30
Gilmore, B., McAuliffe, É., Larkan, F., Conteh, M., Dunne, N., Gaudrault, M., Mollel, H. A., Tumwesigye, N. M., & Vallières, F. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011885 · Cited by 29
“Advocates note that their methodological design better enables the evaluation of complex health interventions compared to quasi-experimental designs.(22) 30”
BackgroundUne synthèse exploratoire du courtage en connaissance en santé publique
Ridde, V., Dagenais, C., & Boileau-Falardeau, M. (2013) · Santé Publique · doi:10.3917/spub.132.0137 · Cited by 26
“Cette démarche évaluative prometteuse reste cependant encore en construction et à notre connaissance n’a pas encore exploré le champ du courtage de connaissances [37, 38].”
BackgroundFick, F., & Muhajarine, N. (2019) · International Journal of Social Research Methodology · doi:10.1080/13645579.2019.1595375 · Cited by 34
Palm, R., & Hochmuth, A. (2020) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2020.103601 · Cited by 33
Pomey, M., de Guise, M., Desforges, M., Bouchard, K., Vialaron, C., Normandin, L., Nelea, M. I., Fortin, I., Ganache, I., Régis, C., et al. (2021) · BMC Health Services Research · doi:10.1186/s12913-020-06009-4 · Cited by 28
Goicolea, I., Vives‐Cases, C., Sebastiån, M. S., Marchal, B., Kegels, G., & Hurtig, A. (2013) · Implementation Science · doi:10.1186/1748-5908-8-36 · Cited by 27
“Over recent years, realist evaluation has been gaining increased recognition within health systems research [45,47,48,50,54-59].”
BackgroundMethodSelf-citationMcHugh, S., Tracey, M., Riordan, F., O’Neill, K., Mays, N., & Kearney, P. M. (2015) · Implementation Science · doi:10.1186/s13012-016-0464-9 · Cited by 27
Byskov, J., Maluka, S. O., Marchal, B., Shayo, E. H., Blystad, A., Bukachi, S. A., Zulu, J. M., Michelo, C., Hurtig, A., & Bloch, P. (2019) · Health Policy and Planning · doi:10.1093/heapol/czz042 · Cited by 26
Self-citationChandler, C., DiLiberto, D., Nayiga, S., Taaka, L., Nabirye, C., Kayendeke, M., Hutchinson, E., Kizito, J., Maiteki‐Sebuguzi, C., Kamya, M. R., et al. (2013) · Implementation Science · doi:10.1186/1748-5908-8-113 · Cited by 25
“There is much variation in the concerns and methods used by those adopting a theory-driven evaluation approach [25,26], although most propose to explicate a theory or model of the programme/intervention and use this to guide and strengthen evaluation questions and analysis [27].”
MethodRenmans, D., Holvoet, N., & Criel, B. (2020) · New Directions for Evaluation · doi:10.1002/ev.20424 · Cited by 26
“However, recent reviews (Lemire et al., this issue; Marchal et al., 2012) show that the analysis of the context is deemed problematic.”
BackgroundHall, T., Goldfeld, S. R., Loftus, H., Honisett, S., Liu, H., de Souza, D., Bailey, C., Reupert, A., Yap, M. B. H., Eapen, V., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-055431 · Cited by 26
Schools’ capacity to absorb aHealthy Schoolapproach into their operations
Deschesnes, M., Drouin, N., Tessier, C., & Couturier, Y. (2014) · Health Education · doi:10.1108/he-10-2013-0054 · Cited by 25
Care Planning: What Works, for Whom, and in What Circumstances? A Rapid Realist Review
Brown, S., Lhussier, M., Dalkin, S. M., & Eaton, S. D. (2018) · Qualitative Health Research · doi:10.1177/1049732318768807 · Cited by 28
Holte, K. A., & Follo, G. (2018) · Safety Science · doi:10.1016/j.ssci.2018.05.020 · Cited by 25
de Andrade, M., & Angelova, N. (2018) · Critical Public Health · doi:10.1080/09581596.2018.1541229 · Cited by 24
“There are also intrinsic methodological challenges in ascribing changes in outcomes to complex interventions (Marchal et al., 2012), not least time and resource constraints (Redfern, Christian, & Norman, 2003; Rycroft-Malone, Fontenla, Bick, & Seers, 2010).”
BackgroundRealistic evaluation as a volunteer tourism supply chain methodology
Eckardt, C., Font, X., & Kimbu, A. N. (2019) · Journal of Sustainable Tourism · doi:10.1080/09669582.2019.1696350 · Cited by 24
“In bridging theory to practice, building these CMO configurations is an essential step to explain the interrelations which operationalise the mechanism (Manzano, 2016; Marchal et al., 2012; Pawson & Manzano-Santaella, 2012), which will determine the potential methods of measurement of why volunteer tourism stakeholder…”
BackgroundInfluentialImplementing health system change: What are the lessons from the African Health Initiative?
Gilson, L. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-s2-s14 · Cited by 24
“This is the aim of “theory driven inquiry” [10].”
BackgroundCleary, S., Erasmus, E., Gilson, L., Michel, C., Gremu, A., Sherr, K., & Olivier, J. (2018) · Health Policy and Planning · doi:10.1093/heapol/czy051 · Cited by 24
Realist analysis of qualitative data in health professions education research
Rees, C. E., Proctor, D. W., Nguyen, V. N. B., Ottrey, E., & Mattick, K. L. (2024) · Medical Education · doi:10.1111/medu.15482 · Cited by 25
“25,33 This has been described as the data reduction (or consolidation) phase to help make sense of voluminous data.”
Kite, J., Indig, D., Mihrshahi, S., Milat, A., & Bauman, A. E. (2015) · Preventive Medicine · doi:10.1016/j.ypmed.2015.08.012 · Cited by 23
Dainty, K. N., Golden, B. R., Hannam, R., Webster, F., Browne, G. B., Mittmann, N., Stern, A., & Zwarenstein, M. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2018.04.006 · Cited by 24
Abayneh, S., Lempp, H. K., Manthorpe, J., & Hanlon, C. (2018) · International Journal of Mental Health Systems · doi:10.1186/s13033-018-0220-4 · Cited by 24
Looking beneath the surface of a preceptor-training programme through a realist evaluation
Hugo, L., & Botma, Y. (2019) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2019.01.005 · Cited by 22
Realist evaluation in times of decolonising global health
Renmans, D., Sarkar, N. D. P., Belle, S. V., Affun‐Adegbulu, C., Marchal, B., & Mukumbang, F. C. (2022) · The International Journal of Health Planning and Management · doi:10.1002/hpm.3530 · Cited by 22
Self-citationRealist evaluation of an enhanced health visiting programme
Doi, L., Jepson, R. G., & Hardie, S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0180569 · Cited by 20
“Realist evaluation argues that in order for evaluation to be useful to decision makers, it needs to explain ‘what works for whom, how, in what circumstances and why’ [9, 15].”
BackgroundHudon, C., Chouinard, M., Aubrey‐Bassler, K., Burge, F., Doucet, S. A., Ramsden, V. R., Brodeur, M., Bush, P. L., Couturier, Y., Dubois, M., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-026433 · Cited by 21
Wynendaele, H., Clays, E., Peeters, E., DeJonghe, Y., Hecke, A. V., & Trybou, J. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-12966-5 · Cited by 24
“…future research might benefit more from adopting hybrid study designs that not only assess effectiveness but also emphasize implementation factors, or from engaging in realist evaluation studies that offer deeper insights into 'what works, for whom, in what circumstances, and why [26, 63].”
Co-production and arts-informed inquiry as creative power for knowledge mobilisation
MacGregor, S., Cooper, A., Searle, M. J., & Kukkonen, T. (2022) · Evidence & Policy · doi:10.1332/174426421x16478737939339 · Cited by 20
Improving Process Evaluations of Health Behavior Interventions
Morgan‐Trimmer, S. (2013) · Evaluation & the Health Professions · doi:10.1177/0163278713497363 · Cited by 20
Nambiar, D., & Sheikh, K. (2016) · Health Systems & Reform · doi:10.1080/23288604.2016.1148802 · Cited by 20
“Such an approach inherently questions the assumption of a linear or direct relationship between participation and various outcomes, at the expense of key contextual factors that influence them.(46,47)”
BackgroundMladovsky, P., & Bâ, M. (2017) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.07.002 · Cited by 20
Dossou, J., Brouwere, V. D., Belle, S. V., & Marchal, B. (2019) · Health Policy and Planning · doi:10.1093/heapol/czz146 · Cited by 20
Self-citationRandell, R., Greenhalgh, J., Hindmarsh, J., Honey, S., Pearman, A. D., Alvarado, N., & Dowding, D. (2019) · Health An Interdisciplinary Journal for the Social Study of Health Illness and Medicine · doi:10.1177/1363459319874115 · Cited by 19
“Realist enquiry is a theory-driven approach that involves eliciting, testing and refining stakeholders’ theories of how an intervention works (Pawson and Tilley, 1997) and has been used for studying the implementation of a number of complex interventions in healthcare (Dalkin et al., 2015; Greenhalgh et al., 2009; Mar…”
BackgroundMonitoring and evaluation for thinking and working politically
Aston, T., Roche, C., Schaaf, M., & Cant, S. (2021) · Evaluation · doi:10.1177/13563890211053028 · Cited by 22
“There is some heterogeneity and debate among realist evaluators about both epistemological and methodological issues, but the description that follows largely reflects areas of consensus (Manzano, 2016; Marchal et al., 2012).”
InfluentialEvaluating impact of clinical guidelines using a realist evaluation framework
Reddy, S., Wakerman, J., Westhorp, G., & Herring, S. E. (2015) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12482 · Cited by 19
Alvarado, N., Honey, S., Greenhalgh, J., Pearman, A. D., Dowding, D., Cope, A., Long, A., Jayne, D. G., Gill, A., Kotzé, A., et al. (2017) · Evaluation · doi:10.1177/1356389017732172 · Cited by 20
“Surfacing realist mechanisms take understanding beyond intervention description into explanation (Pawson and Manzano-Santaella, 2012 ; Marchal et al., 2012).”
BackgroundMethodInfluentialTremblay, D., Touati, N., Poder, T. G., Vasiliadis, H., Bilodeau, K., Berbiche, D., Denis, J., Pomey, M., Hébert, J., Roch, G., et al. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4586-z · Cited by 19
Application of the critical incident technique in refining a realist initial programme theory
Cunningham, Ú., Brún, A. D., & McAuliffe, É. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01016-9 · Cited by 19
Collaborations in volunteer tourism
Eckardt, C., Font, X., & Kimbu, A. N. (2021) · Current Issues in Tourism · doi:10.1080/13683500.2021.1955842 · Cited by 18
Lâm, S., Dodd, W., Wyngaarden, S., Skinner, K. A., Papadopoulos, A., & Harper, S. L. (2021) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2021.102008 · Cited by 18
Graham, F. P., Williman, J. A., Sutherland, L., & Wijninckx, M. (2022) · Child Care Health and Development · doi:10.1111/cch.13028 · Cited by 18
“…evaluation can provide generative explanation for phenomena and help identify patterns linking contexts and mechanisms with particular outcomes (Marchal et al., 2012) and has proven helpful in understanding implementation of complex paediatric rehabilitation interventions (Martin et al., 2020;…”
BackgroundStandardised data on initiatives—STARDIT: Beta version
Nunn, J., Shafee, T. M. A., Chang, S., Stephens, R., Elliott, J., Oliver, S. R., John, D., Smith, M. P., Orr, N., Preston, J., et al. (2022) · Research Involvement and Engagement · doi:10.1186/s40900-022-00363-9 · Cited by 17
Giralt, A. N., Nöstlinger, C., Lee, J. C. L., Salami, O., Lallemant, M. J., Onyango, O., Nyamongo, I. K., & Marchal, B. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014528 · Cited by 18
Self-citationIliffe, S., Davies, N., Manthorpe, J., Crome, P., Ahmedzai, S. H., Vernooij-Dassen, M. J. F. J., & Engels, Y. (2016) · BMC Palliative Care · doi:10.1186/s12904-016-0144-1 · Cited by 21
“Before-and-after studies tend to overestimate effect sizes [56], although this probably applies more to quantitative outcomes than to organisational processes, and realist evaluation has methodological problems that may limit its potential [57].”
BackgroundStoszkowski, J., & Collins, D. J. (2021) · Sport Education and Society · doi:10.1080/13573322.2021.1952180 · Cited by 21
“As such, the impact of the same intervention will likely vary considerably depending on a range of interacting components, and our findings do not provide definitive predictions or a universally valid ‘yes or no’ answer about whether this educational intervention will be effective (Byng et al., 2005; Marchal et al., 2…”
BackgroundInfluentialMcEwan, K., Girling, M., Bate, A., Atkinson, J., Clarke, A., & Dalkin, S. M. (2023) · International Journal of Social Research Methodology · doi:10.1080/13645579.2023.2184920 · Cited by 18
“Programmes/interventions under investigation comprise a set of assumptions held by those who designed and deliver them which ‘explain how and why they expect the intervention to reach its objective(s) and in what conditions’ (Marchal et al., 2012, p. 83).”
BackgroundMethodOrgill, M., Marchal, B., Shung-King, M., Sikuza, L., & Gilson, L. (2021) · BMC Public Health · doi:10.1186/s12889-021-10546-w · Cited by 19
Self-citationTowards an Explanation of the Social Value of Health Systems: An Interpretive Synthesis
Whyle, E. B., & Olivier, J. (2020) · International Journal of Health Policy and Management · doi:10.34172/ijhpm.2020.159 · Cited by 20
Giralt, A. N., Nöstlinger, C., Lee, J. C. L., Salami, O., Lallemant, M. J., Onyango‐Ouma, W., Nyamongo, I. K., & Marchal, B. (2019) · PLoS ONE · doi:10.1371/journal.pone.0220408 · Cited by 16
Self-citationMobilising Q methodology within a realist evaluation: Lessons from an empirical study
Harris, K., Henderson, S., & Wink, B. (2019) · Evaluation · doi:10.1177/1356389019841645 · Cited by 16
“To reiterate the terms of mechanism and context above, Marchal et al. (2012) state that realist evaluators believe causality to be generative in the sense that actors have the potential for change by their very nature (agency).”
BackgroundDossou, J., Belle, S. V., & Marchal, B. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.553980 · Cited by 15
“Formulating the Initial Programme Theory Realist research proceeds in iterative cycles that start and end with the formulation of a programme theory (3, 7).”
MethodSelf-citationReflecting on Reflexivity in Realist Evaluation: A Call to Action
Downey, J., Harris, N., Rybczynska‐Bunt, S., Golder, E., Shearn, K., & Bradley, N. (2024) · International Journal of Qualitative Methods · doi:10.1177/16094069241284206 · Cited by 15
“Realist evaluation is a form of theory driven evaluation that investigates generative causation for its explanatory value (Marchal et al., 2012).”
Munar, W. A., Wahid, S. S., & Curry, L. A. (2018) · Gates Open Research · doi:10.12688/gatesopenres.12782.1 · Cited by 16
Price, D., Edwards, M., Davies, F., Cooper, A., McFadzean, J., Carson-Stevens, A., Cooke, M., Dale, J., Evans, B. A., Harrington, B., et al. (2022) · BMC Emergency Medicine · doi:10.1186/s12873-021-00562-9 · Cited by 15
“We then aimed to integrate these theories as a ‘Programme Theory’ [42] to explain why using primary care clinicians in or alongside EDs may or may not work to improve experience or acceptability of streaming, for whom, and in what specific circumstances.”
MethodEvaluating The Virtues Project as a leadership development programme
Newstead, T., Dawkins, S., Macklin, R., & Martin, A. J. (2020) · Leadership · doi:10.1177/1742715019899845 · Cited by 17
“A MRT sits somewhere between micro hypotheses anticipating correlations between specific variables, and macro theories of unified behaviour, change, and organizing (Marchal et al., 2012).”
BackgroundMethodClark, J. D., & Kearns, A. (2016) · Environment and Planning C Government and Policy · doi:10.1177/0263774x15624923 · Cited by 15
Štreimikienė, D. (2016) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2016.12.017 · Cited by 15
Co-located welfare advice in general practice: A realist qualitative study
Woodhead, C., Collins, H., Lomas, R., & Raine, R. (2017) · Health & Social Care in the Community · doi:10.1111/hsc.12453 · Cited by 15
Sligo, J. L., Roberts, V., Gauld, R. D. C., Villa, L., & Thirlwall, S. (2019) · Health Policy and Technology · doi:10.1016/j.hlpt.2019.08.001 · Cited by 15
Unpacking the Black Box: A realist evaluation of performance management for social services
Tan, H. T. R., & Harvey, G. (2015) · Public Management Review · doi:10.1080/14719037.2015.1112422 · Cited by 14
“The use of realist evaluation has garnered increased attention in the evaluation of health-care policies, programmes and interventions (Byng, Norman, and Redfern 2005; Marchal et al. 2012; Blamey and Mackenzie 2007).”
MethodOwusu-Addo, E., Renzaho, A., & Smith, B. J. (2020) · World Development · doi:10.1016/j.worlddev.2020.104920 · Cited by 13
Sheaff, R., Windle, K., Wistow, G., Ashby, S., Beech, R., Dickinson, A. M., Henderson, C., & Knapp, M. R. J. (2014) · Social Science & Medicine · doi:10.1016/j.socscimed.2014.01.033 · Cited by 14
Hensel, J. M., Shaw, J., Jeffs, L. P., Ivers, N. M., Desveaux, L., Cohen, A., Agarwal, P., Wodchis, W. P., Tepper, J. D., Larsen, D., et al. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-1057-5 · Cited by 15
“Realist Evaluation [26] is growing in international popularity in the health services and policy research community [27], and is proposed for the evaluation of complex health care interventions [26].”
BackgroundFeather, J. (2018) · International Journal of Care Coordination · doi:10.1177/2053434518779753 · Cited by 14
Four decades of overdose prevention centres: lessons for the future from a realist review
Keemink, J. R., Stevens, A., Shirley‐Beavan, S., Khadjesari, Z., & Shorter, G. W. (2025) · Harm Reduction Journal · doi:10.1186/s12954-025-01178-z · Cited by 13
“Realist analysis provides explanations in the form of contingent combinations of contexts, mechanisms, outcomes (CMO) [28].”
Yeates, P., Maluf, A., Kinston, R., Cope, N. A., McCray, G., Cullen, K. M., O’Neill, V., Cole, A., Goodfellow, R., Vallender, R., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-064387 · Cited by 13
“Data will be collected iteratively, interspersed by analysis, 37 through a mixture of observation, individual inter-views 38 and (where feasible) focus groups, supplemented by available process data.”
MethodSystems approaches to public service delivery: methods and frameworks
Mansoor, Z., & Williams, M. J. (2023) · Journal of Public Policy · doi:10.1017/s0143814x23000405 · Cited by 26
“Whereas systems approaches are sometimes perceived as being from a different epistemological tradition than standard impact evaluation methods (e.g. Marchal et al. 2012), we view the underlying epistemology of systems approaches as consistent with that of impact evaluation.”
Successfully developing advanced practitioner roles: policy and practice mechanisms
Officer, T. N., Cumming, J., & McBride‐Henry, K. (2018) · Journal of Health Organization and Management · doi:10.1108/jhom-02-2018-0064 · Cited by 16
Floate, H. J., Marks, G. C., & Durham, J. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028314 · Cited by 18
“Typically, to achieve this differentiation, potential theories (or candidate theories) of the CMOs in which a programme is or will be implemented are generated throughout the review to account for the processes of an intervention that lead to an outcome.(36) CMO configurations and potential theories are then analysed…”
BackgroundRealist evaluation of public engagement and involvement in data-intensive health research
Hobbs, G., & Tully, M. P. (2020) · Research Involvement and Engagement · doi:10.1186/s40900-020-00215-4 · Cited by 12
The health and well-being impacts of a work integration social enterprise from a systems perspective
Joyce, A., Elmes, A., Campbell, P., Moussa, B., Suchowerska, R., Barraket, J., & Carey, G. E. (2021) · Health Promotion International · doi:10.1093/heapro/daab052 · Cited by 13
Gage, H., Family, H. E., Murphy, F., Williams, P., Sutton, J. W., & Taylor, D. (2014) · Journal of Advanced Nursing · doi:10.1111/jan.12527 · Cited by 13
Williams, L., Rycroft‐Malone, J., & Burton, C. R. (2016) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2016.04.012 · Cited by 13
O’Brien, R., Buston, K., Wight, D., McGee, E., White, J., & Henderson, M. (2019) · Trials · doi:10.1186/s13063-019-3395-3 · Cited by 13
“It is important to explore the wider contextual factors that may influence the functioning of ETPB and MB, given that the effects of intervention come about through the interaction of their mechanisms with people and contexts [28].”
Background
Metadata
- Authors
- Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G.
- Affiliations
- ITM-A, Belgium
- BTC, Belgium
- Venue
- Evaluation
- Publisher
- SAGE Publications
- Published
- 2012-04
- Volume
- 18
- Issue
- 2
- Pages
- 192-212
- ISSN
- 1356-3890 (ISSN-L)
- 1461-7153
- OpenAlex
- W2077589521
- Cited by
- 476
- References
- 92
- Citations by year
- 2026: 22 · 2025: 29 · 2024: 41 · 2023: 29 · 2022: 51 · 2021: 64 · 2020: 31 · 2019: 40 · 2018: 37 · 2017: 28 · 2016: 31 · 2015: 34 · 2014: 24 · 2013: 12 · 2012: 3
- FWCI
- 30.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Evaluation (publishedVersion)
- Topics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- realist evaluation, health systems research, mechanisms, context, intervention, theory of change, theory-based evaluation, methodological recommendations, conceptual clarity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
