Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research

Marchal, Bruno; van Belle, Sara; van Olmen, Josefien; Hoerée, Tom; Kegels, Guy

doi:10.1177/1356389012442444

Cited by 476Evaluation and Performance AssessmentHealth Policy Implementation ScienceAdvanced Causal Inference Techniques

Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research

Marchal, Bruno, van Belle, Sara, van Olmen, Josefien, Hoerée, Tom, Kegels, Guy·2012·Evaluation

DOI 10.1177/1356389012442444

Open this paper in CoChat — free account

Abstract

This overview aims to stimulate conceptual and practical discussions to help unlock the full potential of realist evaluation in health systems research. Based on a structured literature search, this review maps how the concepts of realist evaluation are applied in health systems research and which methodological problems are encountered. We found a great diversity in the depth of application of the philosophical concepts, use of terminology and scope of application in the research process. Terms of theory-driven evaluation, theories of change and realist evaluation are often used interchangeably. Diverging views exist regarding the nature of ‘mechanism’ and the difference between mechanism and essential context condition. A lack of methodological guidance was highlighted by many authors. Realist evaluation is slowly gaining traction in health systems research, but more clarity is needed concerning the definitions of mechanisms and context and how the configuration of context, mechanism and intervention can be described and assessed.

Integrity signals

Registry

Crossref · 10.1177/1356389012442444

Cites

93 of 93 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Marchal et al. noted that ToC and realist evaluation were often used together or interchangeably.
“Marchal et al. [18], in a systematic review on realist evaluation in health systems research, also noted that ToC and realist evaluation were often used together or interchangeably.”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [18]
Both a published literature review and critical discussions in the applied literature cite several methodological challenges associated with a RE approach, including deciding what constitutes a mechanism, and differentiating between what is a mechanism and what is context.
“Both a published literature review 112 and critical discussions in the applied literature cite several methodological challenges associated with a RE approach, including deciding what constitutes a mechanism, 127 and indeed differentiating between what is a mechanism and what is context.”
From ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study (2015) · cites this paper as [112]

Social Theory and Social Structure
Ross, E. J., & Merton, R. Κ. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,318 · see citing works
Social Theory and Social Structure.
Bock, K. E., & Merton, R. Κ. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,721 · see citing works
Systems Thinking, Systems Practice
Checkland, P. B. (1981) · Cumulative Index of Computer Aided Architectural Design · Cited by 6,256 · see citing works
Systems thinking, systems practice
(1982) · European Journal of Operational Research · doi:10.1016/0377-2217(82)90206-5 · Cited by 3,579 · see citing works
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190 · see citing works
A set-theoretic approach to organizational configurations
Fiss, P. C. (2007) · Academy of Management Review · doi:10.5465/amr.2007.26586092 · Cited by 2,225 · see citing works
Evidence-based Policy
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791 · see citing works
Critical realism: essential readings
(1999) · Choice Reviews Online · doi:10.5860/choice.36-5017 · Cited by 1,683 · see citing works
Designing and evaluating complex interventions to improve health care
Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L. (2007) · BMJ · doi:10.1136/bmj.39108.379965.be · Cited by 1,331 · see citing works
Theory-driven evaluations
Chen, H.-T. (1990) · Medical Entomology and Zoology · Cited by 1,227 · see citing works
Evidence-Based Public Health: Moving Beyond Randomized Trials
Victora, C. G., Habicht, J.-P., & Bryce, J. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.400 · Cited by 1,056 · see citing works
Mintzberg on management: inside our strange world of organizations
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3961 · Cited by 1,042 · see citing works
Unpacking black boxes: mechanisms and theory building in evaluation
Astbury, B., & Leeuw, F. L. (2010) · American Journal of Evaluation · doi:10.1177/1098214010371972 · Cited by 850 · see citing works
The Science of Improvement
Berwick, D. M. (2008) · JAMA · doi:10.1001/jama.299.10.1182 · Cited by 809 · see citing works
Evidence-based policy: the promise of ‘realist synthesis’
Pawson, R. (2002) · Evaluation · doi:10.1177/135638902401462448 · Cited by 729 · see citing works
Theory‐based evaluation: Past, present, and future
Weiss, C. H. (1997) · New Directions for Evaluation · doi:10.1002/ev.1086 · Cited by 634 · see citing works
From the Editors Beyond Contextualization: Using Context Theories to Narrow the Micro-Macro Gap in Management Research
Bamberger, P. A. (2008) · Academy of Management Journal · doi:10.5465/amj.2008.34789630 · Cited by 595 · see citing works
Theory-driven evaluations
Lipsey, M. W. (1991) · Evaluation and Program Planning · doi:10.1016/0149-7189(91)90027-e · Cited by 592 · see citing works
Research designs for studies evaluating the effectiveness of change and improvement strategies
Eccles, M. (2003) · Quality and Safety in Health Care · doi:10.1136/qhc.12.1.47 · Cited by 586 · see citing works
A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009
Coryn, C. L. S., Noakes, L. A., Westine, C. D., & Schröter, D. C. (2011) · American Journal of Evaluation · doi:10.1177/1098214010389321 · Cited by 531 · see citing works
Mintzberg on management: Inside our strange world of organizations
Petersen, P. B. (1990) · Organizational Dynamics · doi:10.1016/0090-2616(90)90050-y · Cited by 527 · see citing works
Outcome pattern matching and program theory
Trochim, W. M. K. (1989) · Evaluation and Program Planning · doi:10.1016/0149-7189(89)90052-9 · Cited by 481 · see citing works
Evaluating With Sense
Chen, H.-T., & Rossi, P. H. (1983) · Evaluation Review · doi:10.1177/0193841x8300700301 · Cited by 420 · see citing works
Studying the Organisation and Delivery of Health Services: Research Methods
Goodwin, N. (2002) · Health & Social Care in the Community · doi:10.1046/j.1365-2524.2002.03832.x · Cited by 386 · see citing works
Theories of Change and Realistic Evaluation: Peas in a Pod or Apples and Oranges?
Blamey, A., & Mackenzie, M. (2007) · Evaluation · doi:10.1177/1356389007082129 · Cited by 383 · see citing works
Practical Program Evaluation
Chen, H.-T. (2005) · doi:10.4135/9781412985444 · Cited by 365 · see citing works
How Do You Modernize a Health Service? A Realist Evaluation of Whole‐Scale Transformation in London
Greenhalgh, T., Humphrey, C., HUGHES, J., Macfarlane, F., Butler, C., & Pawson, R. (2009) · Milbank Quarterly · doi:10.1111/j.1468-0009.2009.00562.x · Cited by 364 · see citing works
Theory-Based Evaluation and Types of Complexity
Stame, N. (2004) · Evaluation · doi:10.1177/1356389004043135 · Cited by 350 · see citing works
The Importance of External Validity
Steckler, A., & McLeroy, K. R. (2007) · American Journal of Public Health · doi:10.2105/ajph.2007.126847 · Cited by 295 · see citing works
Reconstructing Program Theories: Methods Available and Problems to be Solved
Leeuw, F. L. (2003) · American Journal of Evaluation · doi:10.1016/s1098-2140(02)00271-0 · Cited by 254 · see citing works
Understanding what works—and why—in quality improvement: the need for theory-driven evaluation
Walshe, K. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm004 · Cited by 237 · see citing works
The Contribution of Critical Realism as an Underpinning Philosophy for OR/MS and Systems
Mingers, J. (2000) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2601033 · Cited by 235 · see citing works
The Multi-Goal, Theory-Driven Approach to Evaluation: A Model Linking Basic and Applied Social Science
Chen, H.-T., & Rossi, P. H. (1980) · Social Forces · doi:10.1093/sf/59.1.106 · Cited by 219 · see citing works
Evidence, Understanding and Complexity: Evaluation in Non-Linear Systems
Barnes, M., Matka, E., & Sullivan, H. (2003) · Evaluation · doi:10.1177/13563890030093003 · Cited by 208 · see citing works
Critical realism: a way forward for evaluation research in nursing?
McEvoy, P., & Richards, D. A. (2003) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02730.x · Cited by 203 · see citing works
The theory-driven approach to validity
Chen, H.-T., & Rossi, P. H. (1987) · Evaluation and Program Planning · doi:10.1016/0149-7189(87)90025-5 · Cited by 198 · see citing works
Studying the Organisation and Delivery of Health Services
(2004) · Routledge eBooks · doi:10.4324/9780203481981 · Cited by 193 · see citing works
Using Realistic Evaluation to Evaluate a Practice-level Intervention to Improve Primary Healthcare for Patients with Long-term Mental Illness
Byng, R., Norman, I. J., & Redfern, S. (2005) · Evaluation · doi:10.1177/1356389005053198 · Cited by 187 · see citing works
A realistic evaluation: the case of protocol-based care
Rycroft-Malone, J., Fontenla, M., Bick, D., & Seers, K. (2010) · Implementation Science · doi:10.1186/1748-5908-5-38 · Cited by 173 · see citing works
Reconstructing Program Theories: Methods Available and Problems to be Solved
Leeuw, F. L. (2003) · American Journal of Evaluation · doi:10.1177/109821400302400102 · Cited by 170 · see citing works
A realist evaluation of the management of a well- performing regional hospital in Ghana
Marchal, B., Dedzo, M., & Kegels, G. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-24 · Cited by 156 · see citing works
Self-citation
Realist Evaluation in Practice
Kazi, M. A. F. (2003) · SAGE Publications Ltd eBooks · doi:10.4135/9781849209762 · Cited by 156 · see citing works
Theory‐based evaluation: Reflections ten years on: Theory‐based evaluation: Past, present, and future
Rogers, P., & Weiss, C. H. (2007) · New Directions for Evaluation · doi:10.1002/ev.225 · Cited by 150 · see citing works
New approaches to evaluating community initiatives: Theory, measurement, and analysis
Connell, K. F.-A. A. A. C. K. A. J. P. (1998) · Cited by 136 · see citing works
Strong theory, flexible methods: Evaluating complex community-based initiatives
Judge, K., & Bauld, L. (2001) · Critical Public Health · doi:10.1080/09581590010028237 · Cited by 135 · see citing works
Issues in the theory-driven perspective
Chen, H.-T., & Rossi, P. H. (1989) · Evaluation and Program Planning · doi:10.1016/0149-7189(89)90046-3 · Cited by 132 · see citing works
Realist Evaluation for Practice
Kazi, M. (2003) · The British Journal of Social Work · doi:10.1093/bjsw/33.6.803 · Cited by 130 · see citing works
Constructing Theories of Change: Methods and Sources
Mason, P., & Barnes, M. (2007) · Evaluation · doi:10.1177/1356389007075221 · Cited by 123 · see citing works
Evaluation of health promotion in schools: a realistic evaluation approach using mixed methods
Pommier, J., Guevel, M., & Jourdan, D. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-43 · Cited by 119 · see citing works
Exposing the key functions of a complex intervention for shared care in mental health: case study of a process evaluation
Byng, R., Norman, I. J., Redfern, S., & Jones, R. H. (2008) · BMC Health Services Research · doi:10.1186/1472-6963-8-274 · Cited by 114 · see citing works
A realist study of the mechanisms of cardiac rehabilitation
Clark, A. M., Whelan, H. K., Barbour, R. S., & MACINTYRE, P. D. (2005) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2005.03601.x · Cited by 104 · see citing works
How to develop a theory-driven evaluation design? Lessons learned from an adolescent sexual and reproductive health programme in West Africa
Belle, S. V., Marchal, B., Dubourg, D., & Kegels, G. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-741 · Cited by 95 · see citing works
Self-citation
Complexity and health – yesterday's traditions, tomorrow's future
Sturmberg, J. P., & Martin, C. M. (2009) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2009.01163.x · Cited by 89 · see citing works
Wanted--new methodologies for health service research. Is complexity theory the answer?
Kernick, D. P. (2006) · Family Practice · doi:10.1093/fampra/cml011 · Cited by 89 · see citing works
A theory-driven approach to evaluating quality of nursing care.
Sidani, S., Doran, D., & Mitchell, P. (2004) · Journal of Nursing Scholarship · doi:10.1111/j.1547-5069.2004.04014.x · Cited by 85 · see citing works
Building Collaborative Capacity through `Theories of Change'
Sullivan, H. T., Barnes, M., & Matka, E. (2002) · Evaluation · doi:10.1177/1358902002008002514 · Cited by 85 · see citing works
The Practice and the Theory
Mackenzie, M., & Blamey, A. (2005) · Evaluation · doi:10.1177/1356389005055538 · Cited by 85 · see citing works
Studying the Organisation and Delivery of Health Services
Blackman, T. (2002) · Dialnet (Universidad de la Rioja) · Cited by 84 · see citing works
Philosophy and scientific realism
Archer, M. S., Bhaskar, R., Collier, A., Lawson, T., & Norrie, A. (2013) · doi:10.4324/9781315008592-6 · Cited by 83 · see citing works
Global Health Promotion
Scriven, A. (2011) · doi:10.1093/obo/9780199756797-0007 · Cited by 82 · see citing works
Realist Evaluation in Practice: Health and Social Work
Weir, E., & Kazi, M. A. F. (2007) · Evaluation Journal of Australasia · doi:10.1177/1035719x0700700108 · Cited by 80 · see citing works
The evaluation of health and social care partnerships: an analysis of approaches and synthesis for the future
Dickinson, H. (2006) · Health & Social Care in the Community · doi:10.1111/j.1365-2524.2006.00650.x · Cited by 78 · see citing works
What do we mean by rigorous health-systems research?
MILLS, A. J., Gilson, L., Hanson, K., Palmer, N., & Lagarde, M. (2008) · The Lancet · doi:10.1016/s0140-6736(08)61633-5 · Cited by 77 · see citing works
Implementing accountability for reasonableness framework at district level in Tanzania: a realist evaluation
Maluka, S. O., Kamuzora, P., Sebastiån, M. S., Byskov, J., Ndawi, B., Olsen, Ø. E., & Hurtig, A. (2011) · Implementation Science · doi:10.1186/1748-5908-6-11 · Cited by 74 · see citing works
A realistic approach to the evaluation of the quality management movement in health care systems: a comparison between European and African contexts based on Mintzberg's organizational models
Blaise, P., & Kegels, G. (2004) · The International Journal of Health Planning and Management · doi:10.1002/hpm.769 · Cited by 68 · see citing works
Self-citation
Tackling health inequalities through partnership working: Learning from a realistic evaluation
Evans, D., & Killoran, A. (2000) · Critical Public Health · doi:10.1080/09581590050075899 · Cited by 66 · see citing works
Developing a managed clinical network in palliative care: a realistic evaluation
Tolson, D., McIntosh, J., Loftus, L., & Cormie, P. (2006) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2005.11.027 · Cited by 60 · see citing works
The conceptual framework of the theory-driven perspective
Chen, H.-T. (1989) · Evaluation and Program Planning · doi:10.1016/0149-7189(89)90057-8 · Cited by 58 · see citing works
Evaluating complex public health interventions: theory, methods and scope of realist enquiry
Connelly, J. B. (2007) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2006.00790.x · Cited by 57 · see citing works
Realist Evaluation as a Framework for the Assessment of Teaching About the Improvement of Care
Ogrinc, G. S., & Batalden, P. B. (2009) · Journal of Nursing Education · doi:10.3928/01484834-20091113-08 · Cited by 55 · see citing works
Applying a realist(ic) framework to the evaluation of a new model of emergency department based mental health nursing practice
Wand, T., White, K., & Patching, J. (2010) · Nursing Inquiry · doi:10.1111/j.1440-1800.2010.00488.x · Cited by 53 · see citing works
Turning around an ailing district hospital: a realist evaluation of strategic changes at Ho Municipal Hospital (Ghana)
Marchal, B., Dedzo, M., & Kegels, G. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-787 · Cited by 49 · see citing works
Self-citation
Review: Promoting Realism in Evaluation
Julnes, G., Mark, M. M., & Henry, G. T. (1998) · Evaluation · doi:10.1177/13563899822208743 · Cited by 42 · see citing works
Evaluating change in health care practice: lessons from three studies
Redfern, S., Christian, S., & Norman, I. J. (2003) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2003.00387.x · Cited by 40 · see citing works
A realistic evaluation of fines for hospital discharges: Incorporating the history of programme evaluations in the analysis
Manzano, A. (2011) · Evaluation · doi:10.1177/1356389010389913 · Cited by 36 · see citing works
Quality improvement reports: a new kind of article
Smith, R. J. (2000) · BMJ · doi:10.1136/bmj.321.7274.1428 · Cited by 34 · see citing works
Analysing process in healthcare: the methodological and theoretical challenges
Calnan, M., & Ferlie, E. B. (2003) · Policy & Politics · doi:10.1332/030557303765371672 · Cited by 33 · see citing works
An Application of Theory-Driven Evaluation to a Drop-In Youth Center
Mercier, C., Piat, M., Péladeau, N., & Dagenais, C. (2000) · Evaluation Review · doi:10.1177/0193841x0002400103 · Cited by 30 · see citing works
Is Realist Evaluation a Realistic Approach for Complex Reforms?
Pedersen, L. H., & Rieper, O. (2008) · Evaluation · doi:10.1177/1356389008090856 · Cited by 30 · see citing works
Advancing the development and application of theory-based evaluation in the practice of public health
Cole, G. E. (1999) · American Journal of Evaluation · doi:10.1016/s1098-2140(99)00033-8 · Cited by 29 · see citing works
The validity of conclusions in evaluation research: A further development of Chen and Rossi's theory-driven approach
Costner, H. L. (1989) · Evaluation and Program Planning · doi:10.1016/0149-7189(89)90051-7 · Cited by 28 · see citing works
Advancing the Development and Application of Theory-based Evaluation in the Practice of Public Health
Cole, G. E. (1999) · American Journal of Evaluation · doi:10.1177/109821409902000305 · Cited by 27 · see citing works
Evaluating Urban Regeneration Programmes in Britain
Ho, S. Y. (1999) · Evaluation · doi:10.1177/135638999400830084 · Cited by 24 · see citing works
Realistic Evaluation of an Illicit Drug Deterrence Programme
Leone, L. (2008) · Evaluation · doi:10.1177/1356389007084673 · Cited by 24 · see citing works
A realistic theory of health sector management
Connelly, J. (2000) · Journal of Management in Medicine · doi:10.1108/02689230010362882 · Cited by 23 · see citing works
Using a realist approach to evaluate smoking cessation interventions targeting pregnant women and young people
Douglas, F., Gray, D., & van Teijlingen, E. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-49 · Cited by 22 · see citing works
Getting beyond outcomes: a realist approach to help understand the impact of a nutritional intervention during smoking cessation
Mackenzie, M., Koshy, P., Leslie, W. S., Lean, M. E. J., & Hankey, C. R. (2009) · European Journal of Clinical Nutrition · doi:10.1038/ejcn.2009.38 · Cited by 18 · see citing works
A methodological inquiry into the evaluation of smoking cessation programmes
Kaneko, M. (1999) · Health Education Research · doi:10.1093/her/14.3.433 · Cited by 14 · see citing works
Developing an evidence base for intermediate care delivered by GPs with a special interest.
Kernick, D. P., & Mannion, R. (2005) · PubMed · Cited by 12 · see citing works
Perception de quelques mécanismes favorables à la réduction des inégalités sociales de santé en France
Ridde, V., & Guichard, A. (2011) · Global Health Promotion · doi:10.1177/1757975911412400 · Cited by 10 · see citing works
Realism in evidence based medicine: interpreting the randomised controlled trial
Connelly, J. (2004) · Journal of Health Organization and Management · doi:10.1108/14777260410538861 · Cited by 10 · see citing works
Solving Methodological Challenges Using a Theory-driven Evaluation in the Study of Complex Patient Care
Grocott, P., Cowley, S., & Richardson, A. M. (2002) · Evaluation · doi:10.1177/135638902401462457 · Cited by 7 · see citing works
Evaluating a community oriented primary care program: Lessons learned through a theory-oriented approach
Rodriguez, E., & Mead, J. P. (1997) · Evaluation and Program Planning · doi:10.1016/s0149-7189(96)00051-1 · Cited by 4 · see citing works

Flagged first, then by citations

Sorted by citations

What’s in a mechanism? Development of a key concept in realist evaluation
Dalkin, S. M., Greenhalgh, J., Jones, D., Cunningham, B., & Lhussier, M. (2015) · Implementation Science · doi:10.1186/s13012-015-0237-x · Cited by 896 · see citing works
RAMESES II reporting standards for realist evaluations
Wong, G., Westhorp, G., Manzano, A., Greenhalgh, J., Jagosh, J., & Greenhalgh, T. (2016) · BMC Medicine · doi:10.1186/s12916-016-0643-1 · Cited by 880 · see citing works
The craft of interviewing in realist evaluation
Manzano, A. (2016) · Evaluation · doi:10.1177/1356389016638615 · Cited by 691 · see citing works
Using theory of change to design and evaluate public health interventions: a systematic review
Breuer, E., Lee, L., De Silva, M., & Lund, C. (2015) · Implementation Science · doi:10.1186/s13012-016-0422-6 · Cited by 425 · see citing works
Realist Synthesis for Public Health: Building an Ontologically Deep Understanding of How Programs Work, For Whom, and In Which Contexts
Jagosh, J. (2019) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044451 · Cited by 346 · see citing works
What works for whom in which circumstances? On the need to move beyond the ‘what works?’ question in organizational intervention research
Nielsen, K. M., & Miraglia, M. (2016) · Human Relations · doi:10.1177/0018726716670226 · Cited by 335 · see citing works
Data Analysis and Synthesis Within a Realist Evaluation: Toward More Transparent Methodological Approaches
Gilmore, B., McAuliffe, E., Power, J., & Vallières, F. (2019) · International Journal of Qualitative Methods · doi:10.1177/1609406919859754 · Cited by 296 · see citing works
ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study
Wilson, P., Mathie, E., Keenan, J., McNeilly, E., Goodman, C., Howe, A., Poland, F., Staniszewska, S., Kendall, S., Munday, D., et al. (2015) · Health Services and Delivery Research · doi:10.3310/hsdr03380 · Cited by 297 · see citing works
“Realist evaluation (RE) is one of several theory-based approaches to evaluation developed within the social sciences, which seeks to address dynamism and context, rather than control for them, to explain more adequately how and, importantly, under what circumstances programmes, interventions or innovations will ‘work’…”
The concept of mechanism from a realist approach: a scoping review to facilitate its operationalization in public health program evaluation
Lacouture, A., Breton, É., Guichard, A., & Ridde, V. (2015) · Implementation Science · doi:10.1186/s13012-015-0345-7 · Cited by 295 · see citing works
Understanding ‘context’ in realist evaluation and synthesis
Greenhalgh, J., & Manzano, A. (2021) · International Journal of Social Research Methodology · doi:10.1080/13645579.2021.1918484 · Cited by 285 · see citing works
Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review
Salter, K. L., & Kothari, A. R. (2014) · Implementation Science · doi:10.1186/s13012-014-0115-y · Cited by 282 · see citing works
Achieving successful community engagement: a rapid realist review
de Weger, E., Vooren, N. V., Luijkx, K. G., Baan, C. A., & Drewes, H. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3090-1 · Cited by 232 · see citing works
Building Realist Program Theory for Large Complex and Messy Interventions
Shearn, K., Allmark, P., Piercy, H., & Hirst, J. (2017) · International Journal of Qualitative Methods · doi:10.1177/1609406917741796 · Cited by 215 · see citing works
Protocol—the RAMESES II study: developing guidance and reporting standards for realist evaluation
Greenhalgh, T., Wong, G., Jagosh, J., Greenhalgh, J., Manzano, A., Westhorp, G., & Pawson, R. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008567 · Cited by 193 · see citing works
Quality and reporting standards, resources, training materials and information for realist evaluation: the RAMESES II project
Wong, G., Westhorp, G., Greenhalgh, J., Manzano, A., Jagosh, J., & Greenhalgh, T. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05280 · Cited by 196 · see citing works
Challenges, solutions and future directions in the evaluation of service innovations in health care and public health
Raine, R., Fitzpatrick, R. M., Barratt, H., Bevan, G., Black, N. A., Boaden, R., Bower, P., Campbell, M., Denis, J., Devers, K. J., et al. (2016) · Health Services and Delivery Research · doi:10.3310/hsdr04160 · Cited by 187 · see citing works
Optimising the benefits of community health workers’ unique position between communities and the health sector: A comparative analysis of factors shaping relationships in four countries
Kok, M. C., Ormel, H., Broerse, J. E. W., Kane, S. S., Namakhoma, I., Otiso, L., Sidat, M., Kea, A. Z., Taegtmeyer, M., Theobald, S., et al. (2016) · Global Public Health · doi:10.1080/17441692.2016.1174722 · Cited by 186 · see citing works
People-centred science: strengthening the practice of health policy and systems research
Sheikh, K., George, A. S., & Gilson, L. (2014) · Health Research Policy and Systems · doi:10.1186/1478-4505-12-19 · Cited by 183 · see citing works
A realist evaluation of social prescribing: an exploration into the context and mechanisms underpinning a pathway linking primary care with the voluntary sector
Bertotti, M., Frostick, C., Hutt, P., Sohanpal, R., & Carnes, D. (2017) · Primary Health Care Research & Development · doi:10.1017/s1463423617000706 · Cited by 185 · see citing works
Travelling models and the challenge of pragmatic contexts and practical norms: the case of maternal health
Olivier de Sardan, J.-P., Diarra, A., & Moha, M. (2017) · Health Research Policy and Systems · doi:10.1186/s12961-017-0213-9 · Cited by 167 · see citing works
Evaluating community engagement in global health research: the need for metrics
the Ethics Working Group of the HIV Prevention Trials Network, MacQueen, K. M., Bhan, A., Frohlich, J., Holzer, J., & Sugarman, J. (2015) · BMC Medical Ethics · doi:10.1186/s12910-015-0033-9 · Cited by 157 · see citing works
Elaborating the Context-Mechanism-Outcome configuration (CMOc) in realist evaluation: A critical realist perspective
de Souza, D. (2013) · Evaluation · doi:10.1177/1356389013485194 · Cited by 155 · see citing works
Revitalizing the setting approach – supersettings for sustainable impact in community health promotion
Bloch, P., Toft, U., Reinbach, H. C., Clausen, L. T., Mikkelsen, B. E., Poulsen, K., & Jensen, B. B. (2014) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-014-0118-8 · Cited by 157 · see citing works
A Mobile App to Improve Self-Management of Individuals With Type 2 Diabetes: Qualitative Realist Evaluation
Desveaux, L., Shaw, J., Saragosa, M., Soobiah, C., Marani, H., Hensel, J. M., Agarwal, P., Onabajo, N., Bhatia, R. S., & Jeffs, L. P. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8712 · Cited by 132 · see citing works
What’s in a Realist Configuration? Deciding Which Causal Configurations to Use, How, and Why
de Weger, E., Vooren, N. V., Wong, G., Dalkin, S. M., Marchal, B., Drewes, H., & Baan, C. A. (2020) · International Journal of Qualitative Methods · doi:10.1177/1609406920938577 · Cited by 134 · see citing works
Self-citation
Using narratives to impact health policy-making: a systematic review
Fadlallah, R., El‐Jardali, F., Nomier, M. A., Hemadi, N., Arif, K., Langlois, É. V., & Akl, E. A. (2019) · Health Research Policy and Systems · doi:10.1186/s12961-019-0423-4 · Cited by 131 · see citing works
Realist review
Hunter, R. L., Gorely, T., Beattie, M., & Harris, K. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1969674 · Cited by 123 · see citing works
Understanding how and why audits work in improving the quality of hospital care: A systematic realist review
Hut-Mossel, L., Ahaus, K., Welker, G. A., & Gans, R. O. B. (2021) · PLoS ONE · doi:10.1371/journal.pone.0248677 · Cited by 129 · see citing works
Interventions to mitigate the effects of poverty and inequality on mental health
Wahlbeck, K., Cresswell‐Smith, J., Haaramo, P., & Parkkonen, J. (2017) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-017-1370-4 · Cited by 113 · see citing works
Mechanisms, contexts and points of contention: operationalizing realist-informed research for complex health interventions
Shaw, J., Gray, C. S., Baker, G. R., Denis, J., Breton, M., Gutberg, J., Embuldeniya, G., Carswell, P., Dunham, A., McKillop, A., et al. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0641-4 · Cited by 111 · see citing works
Qualitative research in nursing and health professions regulation
Squires, A., & Dorsen, C. (2018) · Journal of Nursing Regulation · doi:10.1016/s2155-8256(18)30150-9 · Cited by 108 · see citing works
Learning from success—Toward evidence-informed sustainability transitions in communities
Forrest, N., & Wiek, A. (2014) · Environmental Innovation and Societal Transitions · doi:10.1016/j.eist.2014.01.003 · Cited by 104 · see citing works
Examining the Application of Retroductive Theorizing in Realist-Informed Studies
Mukumbang, F. C., Kabongo, E. M., & Eastwood, J. G. (2021) · International Journal of Qualitative Methods · doi:10.1177/16094069211053516 · Cited by 102 · see citing works
Towards an integrated framework for evaluating transformative innovation policy
Haddad, C. R., & Bergek, A. (2022) · Research Policy · doi:10.1016/j.respol.2022.104676 · Cited by 99 · see citing works
Unearthing how, why, for whom and under what health system conditions the antiretroviral treatment adherence club intervention in South Africa works: A realist theory refining approach
Mukumbang, F. C., Marchal, B., Belle, S. V., & van Wyk, B. E. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3150-6 · Cited by 90 · see citing works
Self-citation
Anchoring contextual analysis in health policy and systems research: A narrative review of contextual factors influencing health committees in low and middle income countries.
George, A., Scott, K., Garimella, S., Mondal, S., Ved, R., & Sheikh, K. (2015) · Social Science & Medicine (1967) · doi:10.1016/j.socscimed.2015.03.049 · Cited by 86 · see citing works
What Is This Thing Called a Mechanism? Findings From a Review of Realist Evaluations
Lemire, S. T., Kwako, A., Nielsen, S. B., Christie, C. A., Donaldson, S. I., & Leeuw, F. L. (2020) · New Directions for Evaluation · doi:10.1002/ev.20428 · Cited by 88 · see citing works
Optimal NHS service delivery to care homes: a realist evaluation of the features and mechanisms that support effective working for the continuing care of older people in residential settings
Goodman, C. M., Davies, S. L., Gordon, A. L., Dening, T., Gage, H., Meyer, J. E., Schneider, J. M., Bell, B., Jordan, J., Martin, F. C., et al. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05290 · Cited by 85 · see citing works
Collaboration between parents and SLTs produces optimal outcomes for children attending speech and language therapy: Gathering the evidence
Klatte, I. S., Lyons, R., Davies, K., Harding, S. A., Marshall, J., McKean, C., & Roulstone, S. (2020) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12538 · Cited by 85 · see citing works
Evaluating a major innovation in hospital design: workforce implications and impact on patient and staff experiences of all single room hospital accommodation
Maben, J., Griffiths, P., Penfold, C., Simon, M., Pizzo, E., Anderson, J. E., Robert, G., Hughes, J., Murrells, T., Brearley, S. A., et al. (2015) · Health Services and Delivery Research · doi:10.3310/hsdr03030 · Cited by 82 · see citing works
Explaining the impact of mHealth on maternal and child health care in low- and middle-income countries: a realist synthesis
Kabongo, E. M., Mukumbang, F. C., Delobelle, P., & Nicol, E. (2021) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-021-03684-x · Cited by 85 · see citing works
Advancing the application of systems thinking in health: a realist evaluation of a capacity building programme for district managers in Tumkur, India
Prashanth, N. S., Marchal, B., Devadasan, N., Kegels, G., & Criel, B. (2014) · Health Research Policy and Systems · doi:10.1186/1478-4505-12-42 · Cited by 81 · see citing works
Self-citation
Can “realist” randomised controlled trials be genuinely realist?
Belle, S. V., Wong, G., Westhorp, G., Pearson, M., Emmel, N. D., Manzano, A., & Marchal, B. (2016) · Trials · doi:10.1186/s13063-016-1407-0 · Cited by 77 · see citing works
Self-citation
A realist synthesis of the effect of social accountability interventions on health service providers’ and policymakers’ responsiveness
Lodenstein, E., Dieleman, M. A., Gerretsen, B., & Broerse, J. E. W. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-98 · Cited by 76 · see citing works
From favours to entitlements: community voice and action and health service quality in Zambia
Schaaf, M., Topp, S., & Ngulube, M. N. (2017) · Health Policy and Planning · doi:10.1093/heapol/czx024 · Cited by 67 · see citing works
Communicable disease control programmes and health systems: an analytical approach to sustainability
Shigayeva, A., & Coker, R. J. (2015) · Health Policy and Planning · doi:10.1093/heapol/czu005 · Cited by 63 · see citing works
A realist approach to eliciting the initial programme theory of the antiretroviral treatment adherence club intervention in the Western Cape Province, South Africa
Mukumbang, F. C., Marchal, B., Belle, S. V., & van Wyk, B. E. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0503-0 · Cited by 70 · see citing works
Self-citation
Critical realism in nursing: an emerging approach.
Schiller, C. J. (2016) · Nursing Philosophy · doi:10.1111/nup.12107 · Cited by 84 · see citing works
What Are the Role and Impact of Public-Private Partnerships in Education? A Realist Evaluation of the Chilean Education Quasi-Market
Verger, A., Bonal, X., & Zancajo, A. (2016) · Comparative Education Review · doi:10.1086/685557 · Cited by 70 · see citing works
Evaluating the effectiveness of a multifaceted, multilevel continuous quality improvement program in primary health care: developing a realist theory of change
Schierhout, G., Hains, J., Si, D., Kennedy, C., Cox, R., Kwedza, R., O’Donoghue, L., Fittock, M., Brands, J., Lonergan, K., et al. (2013) · Implementation Science · doi:10.1186/1748-5908-8-119 · Cited by 67 · see citing works
External Validity and Policy Adaptation: From Impact Evaluation to Policy Design
Williams, M. J. (2018) · The World Bank Research Observer · doi:10.1093/wbro/lky010 · Cited by 66 · see citing works
How to phrase critical realist interview questions in applied social science research
Brönnimann, A. (2021) · Journal of Critical Realism · doi:10.1080/14767430.2021.1966719 · Cited by 58 · see citing works
Towards a programme theory for fidelity in the evaluation of complex interventions
Masterson‐Algar, P., Burton, C. R., Rycroft‐Malone, J., Sackley, C. M., & Walker, M. F. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12174 · Cited by 67 · see citing works
Evaluation of an organizational health intervention for low-skilled workers and immigrants
Busch, C., Koch, T., Clasen, J., Winkler, E., & Vowinkel, J. (2017) · Human Relations · doi:10.1177/0018726716682308 · Cited by 66 · see citing works
Combining Theories of Change and Realist Evaluation in practice: Lessons from a research on evaluation study
Rolfe, S. (2019) · Evaluation · doi:10.1177/1356389019835229 · Cited by 68 · see citing works
Person‐centred care in primary care: What works for whom, how and in what circumstances?
Ahmed, A., van den Muijsenbergh, M. E. T. C., & Vrijhoef, H. J. (2022) · Health & Social Care in the Community · doi:10.1111/hsc.13913 · Cited by 66 · see citing works
Towards an agenda for implementation science in global health: there is nothing more practical than good (social science) theories
Belle, S. V., van de Pas, R., & Marchal, B. (2017) · BMJ Global Health · doi:10.1136/bmjgh-2016-000181 · Cited by 65 · see citing works
Self-citation
Optimal healthcare delivery to care homes in the UK: a realist evaluation of what supports effective working to improve healthcare outcomes
Gordon, A. L., Goodman, C. M., Davies, S. L., Dening, T., Gage, H., Meyer, J. E., Schneider, J. M., Bell, B., Jordan, J., Martin, F. C., et al. (2017) · Age and Ageing · doi:10.1093/ageing/afx195 · Cited by 65 · see citing works
Making implementation science more real
Sarkies, M., Francis‐Auton, E., Long, J. C., Pomare, C., Hardwick, R., & Braithwaite, J. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01661-2 · Cited by 66 · see citing works
The uncritical realism of realist evaluation
Porter, S. (2015) · Evaluation · doi:10.1177/1356389014566134 · Cited by 67 · see citing works
Unpacking context in realist evaluations: Findings from a comprehensive review
Nielsen, S. B., Lemire, S. T., & Tangsig, S. (2021) · Evaluation · doi:10.1177/13563890211053032 · Cited by 63 · see citing works
Supporting general hospital staff to provide dementia sensitive care: A realist evaluation
Handley, M., Bunn, F., & Goodman, C. M. (2018) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2018.10.004 · Cited by 62 · see citing works
Implications of the Changing Conversation About Causality for Evaluators
Gates, E., & Dyson, L. (2016) · American Journal of Evaluation · doi:10.1177/1098214016644068 · Cited by 64 · see citing works
Methods in realist evaluation: A mapping review
Renmans, D., & Castellano Pleguezuelo, V. (2023) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2022.102209 · Cited by 68 · see citing works
Bringing critical realism to nursing practice: Roy Bhaskar's contribution
Williams, L., Rycroft‐Malone, J., & Burton, C. R. (2016) · Nursing Philosophy · doi:10.1111/nup.12130 · Cited by 78 · see citing works
‘The government cannot do it all alone’: realist analysis of the minutes of community health committee meetings in Nigeria
Abimbola, S., Molemodile, S. K., Okonkwo, O. A., Negin, J., Jan, S., & Martiniuk, A. (2015) · Health Policy and Planning · doi:10.1093/heapol/czv066 · Cited by 58 · see citing works
Integration of robotic surgery into routine practice and impacts on communication, collaboration, and decision making: a realist process evaluation protocol
Randell, R., Greenhalgh, J., Hindmarsh, J., Dowding, D., Jayne, D. G., Pearman, A. D., Gardner, P. H., Croft, J., & Kotzé, A. (2014) · Implementation Science · doi:10.1186/1748-5908-9-52 · Cited by 60 · see citing works
Realist evaluation of the antiretroviral treatment adherence club programme in selected primary healthcare facilities in the metropolitan area of Western Cape Province, South Africa: a study protocol
Mukumbang, F. C., Belle, S. V., Marchal, B., & van Wyk, B. E. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-009977 · Cited by 59 · see citing works
Self-citation
L’approche réaliste pour l’évaluation de programmes et la revue systématique
Robert, É., & Ridde, V. (2014) · Mesure et évaluation en éducation · doi:10.7202/1025741ar · Cited by 59 · see citing works
Cost and impact of policies to remove and reduce fees for obstetric care in Benin, Burkina Faso, Mali and Morocco
On behalf of the FEMHealth team, Witter, S., Boukhalfa, C., Cresswell, J. A., Daou, Z., Filippi, V., Ganaba, R., Goufodji, S. B., Lange, I. L., Marchal, B., et al. (2016) · International Journal for Equity in Health · doi:10.1186/s12939-016-0412-y · Cited by 58 · see citing works
Self-citation
Labour inspections and the prevention of psychosocial risks at work: A realist synthesis
Weissbrodt, R., & Giauque, D. (2017) · Safety Science · doi:10.1016/j.ssci.2017.02.012 · Cited by 58 · see citing works
An overview of realist evaluation for simulation-based education
Graham, A., & McAleer, S. (2018) · Advances in Simulation · doi:10.1186/s41077-018-0073-6 · Cited by 57 · see citing works
Putting context centre stage: evidence from a systems evaluation of an area based empowerment initiative in England
Orton, L., Halliday, E. C., Collins, M., Egan, M., Lewis, S., Ponsford, R., Powell, K., Salway, S., Townsend, A. F., Whitehead, M. M., et al. (2016) · Critical Public Health · doi:10.1080/09581596.2016.1250868 · Cited by 56 · see citing works
What works in ‘real life’ to facilitate home deaths and fewer hospital admissions for those at end of life?: results from a realist evaluation of new palliative care services in two English counties
Wye, L., Lasseter, G., Percival, J., Duncan, L., Simmonds, B., & Purdy, S. (2014) · BMC Palliative Care · doi:10.1186/1472-684x-13-37 · Cited by 55 · see citing works
‘Same, but different’: A mixed-methods realist evaluation of a cluster-randomized controlled participatory organizational intervention
Abildgaard, J. S., Nielsen, K. M., Wåhlin-Jacobsen, C. D., Maltesen, T., Christensen, K. B., & Holtermann, A. (2019) · Human Relations · doi:10.1177/0018726719866896 · Cited by 54 · see citing works
A realist evaluation of student use of a virtual reality smartphone application in undergraduate legal education
McFaul, H., & FitzGerald, E. (2019) · British Journal of Educational Technology · doi:10.1111/bjet.12850 · Cited by 59 · see citing works
First Things First: How to Elicit the Initial Program Theory for a Realist Evaluation of Complex Integrated Care Programs
SMEETS, R. G. M., HERTROIJS, D. F. L., MUKUMBANG, F. C., KROESE, M. E. A. L., RUWAARD, D., & ELISSEN, A. M. J. (2022) · The Milbank Quarterly · doi:10.1111/1468-0009.12543 · Cited by 53 · see citing works
Understanding the motivation and performance of community health volunteers involved in the delivery of health programmes in Kampala, Uganda: a realist evaluation
Vareilles, G., Marchal, B., Kane, S. S., Petrič, T., Pictet, G., & Pommier, J. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008614 · Cited by 53 · see citing works
Self-citation
How Can Autonomy Be Maintained and Informal Care Improved for People With Dementia Living in Residential Care Facilities: A Systematic Literature Review
Boumans, J., van Boekel, L. C., Baan, C. A., & Luijkx, K. G. (2018) · The Gerontologist · doi:10.1093/geront/gny096 · Cited by 51 · see citing works
Realist evaluations in low- and middle-income countries: reflections and recommendations from the experiences of a foreign researcher
Gilmore, B. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2019-001638 · Cited by 55 · see citing works
The contributions of scientific realism and critical realism to realist evaluation
Mukumbang, F. C., de Souza, D., & Eastwood, J. G. (2023) · Journal of Critical Realism · doi:10.1080/14767430.2023.2217052 · Cited by 59 · see citing works
The effect of leadership on public service motivation: a multiple embedded case study in Morocco
Belrhiti, Z., Damme, W. V., Belalia, A., & Marchal, B. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-033010 · Cited by 49 · see citing works
Self-citation
A realist process evaluation of robot-assisted surgery: integration into routine practice and impacts on communication, collaboration and decision-making
Randell, R., Honey, S., Hindmarsh, J., Alvarado, N., Greenhalgh, J., Pearman, A. D., Long, A., Cope, A., Gill, A., Gardner, P. H., et al. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05200 · Cited by 48 · see citing works
Action leveraging evidence to reduce perinatal mortality and morbidity (ALERT): study protocol for a stepped-wedge cluster-randomised trial in Benin, Malawi, Tanzania and Uganda
Akuze, J., Annerstedt, K. S., Beňová, L., Chipeta, E. K., Dossou, J., Groß, M. M., Kidanto, H. L., Marchal, B., Alvesson, H. M., Pembe, A. B., et al. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-07155-z · Cited by 44 · see citing works
Self-citation
Challenges related to the analytical process in realist evaluation and latest developments on the use of NVivo from a realist perspective
Bergeron, D. A., & Gaboury, I. (2019) · International Journal of Social Research Methodology · doi:10.1080/13645579.2019.1697167 · Cited by 47 · see citing works
Evaluation of Capacity-Building Program of District Health Managers in India: A Contextualized Theoretical Framework
Prashanth, N. S., Marchal, B., Kegels, G., & Criel, B. (2014) · Frontiers in Public Health · doi:10.3389/fpubh.2014.00089 · Cited by 46 · see citing works
Self-citation
Realist review protocol for understanding the real-world barriers and enablers to practitioners implementing self-management support to people living with and beyond cancer
Kantilal, K., Hardeman, W., Whiteside, H., Karapanagiotou, E. M., Small, M., & Bhattacharya, D. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-037636 · Cited by 45 · see citing works
Towards a framework for enhancing the performance of NHS boards: a synthesis of the evidence about board governance, board effectiveness and board development
Chambers, N., Harvey, G., Mannion, R., Bond, J., & Marshall, J. (2013) · Health Services and Delivery Research · doi:10.3310/hsdr01060 · Cited by 47 · see citing works
Contexts, Mechanisms, and Outcomes That Matter in Dutch Community-Based Physical Activity Programs Targeting Socially Vulnerable Groups
Herens, M. C., Wagemakers, A., Vaandrager, L., van Ophem, J., & Koelen, M. A. (2016) · Evaluation & the Health Professions · doi:10.1177/0163278716652940 · Cited by 45 · see citing works
Exploring the use of Soft Systems Methodology with realist approaches: A novel way to map programme complexity and develop and refine programme theory
Dalkin, S. M., Lhussier, M., Williams, L., Burton, C. R., & Rycroft‐Malone, J. (2018) · Evaluation · doi:10.1177/1356389017749036 · Cited by 50 · see citing works
Implementing large-system, value-based healthcare initiatives: a realist study protocol for seven natural experiments
Sarkies, M., Francis‐Auton, E., Long, J. C., Partington, A., Pomare, C., Nguyen, H. M., Wu, W., Westbrook, J. I., Day, R. O., Lévesque, J., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-044049 · Cited by 44 · see citing works
Methodological reflections on using qualitative research to explore the causal mechanisms of complex health interventions
Bonell, C., Warren, E., & Meléndez-Torres, G. (2022) · Evaluation · doi:10.1177/13563890221086309 · Cited by 40 · see citing works
Realist evaluation: an immanent critique
Porter, S. (2015) · Nursing Philosophy · doi:10.1111/nup.12100 · Cited by 44 · see citing works
What Works for whom in which Circumstances? An Integrated Realist Evaluation Model for Organisational Interventions
Roodbari, H., Nielsen, K. M., & Axtell, C. M. (2023) · Scandinavian Journal of Work and Organizational Psychology · doi:10.16993/sjwop.171 · Cited by 42 · see citing works
Mechanisms for achieving adolescent-friendly services in Ecuador: a realist evaluation approach
Goicolea, I., Coe, A., Hurtig, A., & Sebastiån, M. S. (2012) · Global Health Action · doi:10.3402/gha.v5i0.18748 · Cited by 42 · see citing works
Integrating palliative care within acute stroke services: developing a programme theory of patient and family needs, preferences and staff perspectives
Burton, C. R., & Payne, S. A. (2012) · BMC Palliative Care · doi:10.1186/1472-684x-11-22 · Cited by 41 · see citing works
The SMART personalised self-management system for congestive heart failure: results of a realist evaluation
Bartlett, Y. K., Haywood, A., Bentley, C. L., Parker, J. C., Hawley, M., Mountain, G. A., & Mawson, S. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-014-0109-3 · Cited by 40 · see citing works
Protocol: Realist synthesis of the impact of unemployment insurance policies on poverty and health
Molnar, A., O'Campo, P. J., Ng, E., Mitchell, C., Muntaner, C., Renahy, É., John, A. S., & Shankardass, K. (2014) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2014.09.002 · Cited by 39 · see citing works
Enabling conditions for effective marine spatial planning
Zuercher, R., Ban, N. C., Flannery, W., Guerry, A. D., Halpern, B. S., Magris, R. A., Mahajan, S. L., Motzer, N., Spalding, A. K., Stelzenmüller, V., et al. (2022) · Marine Policy · doi:10.1016/j.marpol.2022.105141 · Cited by 38 · see citing works
Power dynamics and intersectoral collaboration for health in low- and middle-income countries: a realist review
Aivalli, P., Dada, S., Gilmore, B., Srinivas, P. N., & Brún, A. D. (2025) · Health Policy and Planning · doi:10.1093/heapol/czaf022 · Cited by 52 · see citing works

How to do (or not to do) realist evaluations to advance theory, practice, and justice in health systems research
Aivalli, P., Hebbar, P. B., Mbachu, C. O., Srinivas, P. N., Abímbọ́lá, Ṣ., & Dada, S. (2025) · Health Policy and Planning · doi:10.1093/heapol/czaf080 · Cited by 5 · see citing works
Unpacking context in realist evaluations: Findings from a comprehensive review
Nielsen, S. B., Lemire, S. T., & Tangsig, S. (2021) · Evaluation · doi:10.1177/13563890211053032 · Cited by 63 · see citing works
Realist evaluation - promise, problems and practicalities
O’Halloran, P. D., Porter, S., Higgins, A., McConnell, T., Burns, F., & McGaughey, J. (2013) · Research Portal (Queen's University Belfast)
The concept of mechanism from a realist approach: a scoping review to facilitate its operationalization in public health program evaluation
Lacouture, A., Breton, É., Guichard, A., & Ridde, V. (2015) · Implementation Science · doi:10.1186/s13012-015-0345-7 · Cited by 295 · see citing works
The realist approach to evaluation research: an introduction
Hewitt, G., Sims, S., & Harris, R. G. (2012) · International Journal of Therapy and Rehabilitation · doi:10.12968/ijtr.2012.19.5.250 · Cited by 100 · see citing works

Authors: Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G.
Affiliations: ITM-A, Belgium
BTC, Belgium
ORCID: Marchal, Bruno
Kegels, Guy
Venue: Evaluation
Publisher: SAGE Publications
Published: 2012-04
Volume: 18
Issue: 2
Pages: 192-212
ISSN: 1356-3890 (ISSN-L)
1461-7153
DOI: 10.1177/1356389012442444
OpenAlex: W2077589521
Cited by: 476
References: 92
Citations by year: 2026: 22 · 2025: 29 · 2024: 41 · 2023: 29 · 2022: 51 · 2021: 64 · 2020: 31 · 2019: 40 · 2018: 37 · 2017: 28 · 2016: 31 · 2015: 34 · 2014: 24 · 2013: 12 · 2012: 3
FWCI: 30.83
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Evaluation (publishedVersion)
Topics: Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords: realist evaluation, health systems research, mechanisms, context, intervention, theory of change, theory-based evaluation, methodological recommendations, conceptual clarity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks