First Things First: How to Elicit the Initial Program Theory for a Realist Evaluation of Complex Integrated Care Programs
Open accessAbstract
Policy Points: Realist evaluation (RE) is an emerging and promising research approach for evaluating integrated care, addressing what works, how, for whom, and in what circumstances. The rich philosophical foundation of RE, critical realism, can help to systematically unravel an integrated care program's initial theory prior to implementation, as a first step within RE. RE can be considered a robust methodological asset in integrated care research by facilitating a deeper level of insight into program functioning than traditional forms of evaluation do and by shaping a realist‐informed monitoring and evaluation process. Context: The complexity of integrated care and the need for transferable evaluation insights ask for a suitable evaluation paradigm. Realist evaluation (RE), underpinned by the philosophy of critical realism, is a theory‐driven approach that addresses what works, how, for whom, and in what circumstances. The current study illustrates the process needed for RE's first step: eliciting the initial program theory (IPT). The TARGET program, a Dutch primary care initiative to facilitate more integrated care for chronically ill patients, i.e., care that is efficient, tailored, and holistic, was taken as a real‐world case. Methods: An RE approach informed the phased IPT elicitation: (1) identifying an abstract theory framework; (2) formulating the preliminary IPT, building on the abstract theory and informed by previous scientific studies that underpin TARGET; and (3) refining the preliminary IPT, informed by RE expert interviews (n = 7). An RE heuristic tool, specifying the interplay between intervention‐context‐actors‐mechanisms‐outcomes (ICAMO) and retroductive reasoning, was applied to synthesize the underlying theory of individual TARGET components into TARGET's IPT. Findings: Separate but related IPTs were identified for the two main types of actors involved in TARGET: primary care professionals (PCPs) and patients. For both actors, two sorts of mechanisms are assumed to be activated by TARGET, which—via instrumental outcomes—contribute to long‐term quadruple aim targets. The first is confidence to enhance PCPs’ person‐centered conversational skills and to increase patients’ active engagement in TARGET. The second is mutual trust, between PCPs and patients and between PCPs and their network partners. A supportive context is assumed crucial for activating these mechanisms—for example, sufficient resources to invest in integrated care. Conclusions: Although the IPT elicitation process is time intensive and requires a mind shift, it facilitates a deeper insight into program functioning than accommodated by the prevailing experimental designs in integrated care. Furthermore, the design of a realist‐informed evaluation process can be informed by the IPT.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · see citing works
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190 · see citing works
Jagosh, J., Bush, P. L., Salsberg, J., Macaulay, A. C., Greenhalgh, T., Wong, G., Cargo, M., Green, L. W., Herbert, C. P., & Pluye, P. (2015) · BMC Public Health · doi:10.1186/s12889-015-1949-1 · Cited by 758 · see citing works
Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G. (2012) · Evaluation · doi:10.1177/1356389012442444 · Cited by 476 · see citing works
Flagged first, then by citations
- Authors
- SMEETS, R. G. M., HERTROIJS, D. F. L., MUKUMBANG, F. C., KROESE, M. E. A. L., RUWAARD, D., & ELISSEN, A. M. J.
- Affiliations
- Maastricht University (NL)
- University of Washington (US)
- Corresponding author
- SMEETS, ROWAN G. M.
- ORCID
- Venue
- The Milbank Quarterly
- Publisher
- Wiley
- Published
- 2021-11-23
- Volume
- 100
- Issue
- 1
- Pages
- 151-189
- ISSN
- 0887-378X (ISSN-L)
- 1468-0009
- PMC
- PMC8932628
- OpenAlex
- W3217410956
- Cited by
- 53
- References
- 4
- Citations by year
- 2026: 9 · 2025: 13 · 2024: 16 · 2023: 7 · 2022: 7 · 2021: 1
- FWCI
- 4.46
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Milbank Quarterly (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, cc-by)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- Achmea
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- realist evaluation, critical realism, integrated care, target programs, chronic disease, primary care, person-centred care, patient engagement, Quadruple Aim, complex interventions, expert interviews
- MeSH terms
- Communication, Humans, Research Design, Program Evaluation, Delivery of Health Care, Integrated
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review