Cited by 53Evaluation and Performance AssessmentHealth Policy Implementation ScienceAdvanced Causal Inference Techniques

First Things First: How to Elicit the Initial Program Theory for a Realist Evaluation of Complex Integrated Care Programs

Open access
SMEETS, ROWAN G. M., HERTROIJS, DORIJN F. L., MUKUMBANG, FERDINAND C., KROESE, MARIËLLE E. A. L., RUWAARD, DIRK, ELISSEN, ARIANNE M. J.·2022·The Milbank Quarterly

Abstract

Policy Points: Realist evaluation (RE) is an emerging and promising research approach for evaluating integrated care, addressing what works, how, for whom, and in what circumstances. The rich philosophical foundation of RE, critical realism, can help to systematically unravel an integrated care program's initial theory prior to implementation, as a first step within RE. RE can be considered a robust methodological asset in integrated care research by facilitating a deeper level of insight into program functioning than traditional forms of evaluation do and by shaping a realist‐informed monitoring and evaluation process. Context: The complexity of integrated care and the need for transferable evaluation insights ask for a suitable evaluation paradigm. Realist evaluation (RE), underpinned by the philosophy of critical realism, is a theory‐driven approach that addresses what works, how, for whom, and in what circumstances. The current study illustrates the process needed for RE's first step: eliciting the initial program theory (IPT). The TARGET program, a Dutch primary care initiative to facilitate more integrated care for chronically ill patients, i.e., care that is efficient, tailored, and holistic, was taken as a real‐world case. Methods: An RE approach informed the phased IPT elicitation: (1) identifying an abstract theory framework; (2) formulating the preliminary IPT, building on the abstract theory and informed by previous scientific studies that underpin TARGET; and (3) refining the preliminary IPT, informed by RE expert interviews (n = 7). An RE heuristic tool, specifying the interplay between intervention‐context‐actors‐mechanisms‐outcomes (ICAMO) and retroductive reasoning, was applied to synthesize the underlying theory of individual TARGET components into TARGET's IPT. Findings: Separate but related IPTs were identified for the two main types of actors involved in TARGET: primary care professionals (PCPs) and patients. For both actors, two sorts of mechanisms are assumed to be activated by TARGET, which—via instrumental outcomes—contribute to long‐term quadruple aim targets. The first is confidence to enhance PCPs’ person‐centered conversational skills and to increase patients’ active engagement in TARGET. The second is mutual trust, between PCPs and patients and between PCPs and their network partners. A supportive context is assumed crucial for activating these mechanisms—for example, sufficient resources to invest in integrated care. Conclusions: Although the IPT elicitation process is time intensive and requires a mind shift, it facilitates a deeper insight into program functioning than accommodated by the prevailing experimental designs in integrated care. Furthermore, the design of a realist‐informed evaluation process can be informed by the IPT.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/1468-0009.12543

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Authors
SMEETS, R. G. M., HERTROIJS, D. F. L., MUKUMBANG, F. C., KROESE, M. E. A. L., RUWAARD, D., & ELISSEN, A. M. J.
Affiliations
  • Maastricht University (NL)
  • University of Washington (US)
Corresponding author
SMEETS, ROWAN G. M.
Venue
The Milbank Quarterly
Publisher
Wiley
Published
2021-11-23
Volume
100
Issue
1
Pages
151-189
ISSN
  • 0887-378X (ISSN-L)
  • 1468-0009
OpenAlex
W3217410956
Cited by
53
References
4
Citations by year
2026: 9 · 2025: 13 · 2024: 16 · 2023: 7 · 2022: 7 · 2021: 1
FWCI
4.46
Type
journal article
Language
en
Open access
Locations
  • Milbank Quarterly (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Publications (Maastricht University) (publishedVersion, cc-by)
  • Research Publications (Maastricht University) (submittedVersion, other-oa)
Funders
Achmea
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords
realist evaluation, critical realism, integrated care, target programs, chronic disease, primary care, person-centred care, patient engagement, Quadruple Aim, complex interventions, expert interviews
MeSH terms
Communication, Humans, Research Design, Program Evaluation, Delivery of Health Care, Integrated

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review