The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research
Abstract
This overview aims to stimulate conceptual and practical discussions to help unlock the full potential of realist evaluation in health systems research. Based on a structured literature search, this review maps how the concepts of realist evaluation are applied in health systems research and which methodological problems are encountered. We found a great diversity in the depth of application of the philosophical concepts, use of terminology and scope of application in the research process. Terms of theory-driven evaluation, theories of change and realist evaluation are often used interchangeably. Diverging views exist regarding the nature of ‘mechanism’ and the difference between mechanism and essential context condition. A lack of methodological guidance was highlighted by many authors. Realist evaluation is slowly gaining traction in health systems research, but more clarity is needed concerning the definitions of mechanisms and context and how the configuration of context, mechanism and intervention can be described and assessed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Marchal et al. [18], in a systematic review on realist evaluation in health systems research, also noted that ToC and realist evaluation were often used together or interchangeably.”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [18]
In the citing paper
“Both a published literature review 112 and critical discussions in the applied literature cite several methodological challenges associated with a RE approach, including deciding what constitutes a mechanism, 127 and indeed differentiating between what is a mechanism and what is context.”
From ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study (2015) · cites this paper as [112]
Citations
Cites
Social Theory and Social Structure
Ross, E. J., & Merton, R. Κ. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,318
Social Theory and Social Structure.
Bock, K. E., & Merton, R. Κ. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,687
Systems Thinking, Systems Practice
Checkland, P. B. (1981) · Cumulative Index of Computer Aided Architectural Design · Cited by 6,256
Systems thinking, systems practice
(1982) · European Journal of Operational Research · doi:10.1016/0377-2217(82)90206-5 · Cited by 3,579
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190
A set-theoretic approach to organizational configurations
Fiss, P. C. (2007) · Academy of Management Review · doi:10.5465/amr.2007.26586092 · Cited by 2,225
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
Critical realism: essential readings
(1999) · Choice Reviews Online · doi:10.5860/choice.36-5017 · Cited by 1,683
Designing and evaluating complex interventions to improve health care
Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L. (2007) · BMJ · doi:10.1136/bmj.39108.379965.be · Cited by 1,331
Chen, H.-T. (1990) · Medical Entomology and Zoology · Cited by 1,227
Research works printed but not linked: 4
- Connell JP, 1995, New Approaches To Evaluating Community Initiatives. Concepts, Methods, and Contexts
- Kernick D, 2005, British Journal of Medical Practice, 55, 908
- Pawson R, 1997, Realistic Evaluation
- Weiss C, 1995, New Approaches to Evaluating Comprehensive Community Initiatives: Concepts, Methods and Contexts
Cited by
Sorted by citations
What’s in a mechanism? Development of a key concept in realist evaluation
Dalkin, S. M., Greenhalgh, J., Jones, D., Cunningham, B., & Lhussier, M. (2015) · Implementation Science · doi:10.1186/s13012-015-0237-x · Cited by 896
RAMESES II reporting standards for realist evaluations
Wong, G., Westhorp, G., Manzano, A., Greenhalgh, J., Jagosh, J., & Greenhalgh, T. (2016) · BMC Medicine · doi:10.1186/s12916-016-0643-1 · Cited by 880
“Further, as noted in Item 5 above, the evaluation question(s), scope and design may evolve over the course of the evaluation [19].”
BackgroundInfluentialThe craft of interviewing in realist evaluation
Manzano, A. (2016) · Evaluation · doi:10.1177/1356389016638615 · Cited by 691
Using theory of change to design and evaluate public health interventions: a systematic review
Breuer, E., Lee, L., De Silva, M., & Lund, C. (2015) · Implementation Science · doi:10.1186/s13012-016-0422-6 · Cited by 425
“Despite some fundamental differences in their theoretical underpinnings, many of these approaches are used interchangeably or together [18, 20].”
BackgroundInfluentialJagosh, J. (2019) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044451 · Cited by 346
Metadata
- Authors
- Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G.
- Affiliations
- ITM-A, Belgium
- BTC, Belgium
- Venue
- Evaluation
- Publisher
- SAGE Publications
- Published
- 2012-04
- Volume
- 18
- Issue
- 2
- Pages
- 192-212
- ISSN
- 1356-3890 (ISSN-L)
- 1461-7153
- OpenAlex
- W2077589521
- Cited by
- 476
- References
- 92
- Citations by year
- 2026: 22 · 2025: 29 · 2024: 41 · 2023: 29 · 2022: 51 · 2021: 64 · 2020: 31 · 2019: 40 · 2018: 37 · 2017: 28 · 2016: 31 · 2015: 34 · 2014: 24 · 2013: 12 · 2012: 3
- FWCI
- 30.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Evaluation (publishedVersion)
- Topics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- realist evaluation, health systems research, mechanisms, context, intervention, theory of change, theory-based evaluation, methodological recommendations, conceptual clarity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
