The contribution of process tracing to theory-based evaluations of complex aid instruments
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
This article focuses on methodological challenges in evaluating complex program aid interventions like budget support. We show that recent innovations in process-tracing methodology can help solve the identified challenges and increase the strength of causal inference made when using case studies in demanding settings. For the specific task of evaluating the governance effectiveness of budget support interventions, we developed a more fine-grained causal mechanism for a sub-set of the comprehensive program theory of budget support. Moreover, based on the informal use of Bayesian logic, we have elaborated on how to increase the conclusiveness of empirical evidence for one part of the theorized causal mechanism. We argue that by establishing an explicit theorized mechanism prior to empirical research and by critically judging our evidence according to an informal Bayesian logic we can remedy some of the problems at hand in much case-study research and increase the inferential leverage in complex within-case evaluation studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Machamer, P., Darden, L., & Craver, C. F. (2000) · Philosophy of Science · doi:10.1086/392759 · Cited by 3,179 · cited 2×
“Entities are what engage in activities, where the activities are the producers of change or what transmits causal forces through a mechanism (Machamer et al., 2000).”
BackgroundProcess-Tracing Methods: Foundations and Guidelines
Beach, D., & Pedersen, R. (2013) · doi:10.3998/mpub.2556282 · Cited by 1,140
Self-citationProcess-Tracing Methods: Foundations and Guidelines
(2013) · Choice Reviews Online · doi:10.5860/choice.51-1711 · Cited by 921 · cited 4×
“However, when engaging in PT we typically focus on the link(s) between a cause (X) and a particular outcome instead of multiple different outcomes (Beach and Pedersen, 2013: 14–16).”
BackgroundMethodInfluentialContext and Causal Mechanisms in Political Analysis
Falleti, T. G., & Lynch, J. F. (2009) · Comparative Political Studies · doi:10.1177/0010414009331724 · Cited by 909
Bunge, M. (1997) · Philosophy of the Social Sciences · doi:10.1177/004839319702700402 · Cited by 620
Mechanisms in the Analysis of Social Macro-Phenomena
Mayntz, R. (2004) · Philosophy of the Social Sciences · doi:10.1177/0048393103262552 · Cited by 614
Process Tracing: From Metaphor to Analytic Tool
(2014) · Cambridge University Press · doi:10.1017/cbo9781139858472 · Cited by 495
Contribution analysis: Coming of age?
Mayne, J. (2012) · Evaluation · doi:10.1177/1356389012451663 · Cited by 387 · cited 2×
“In CA, stakeholders are interviewed to find out whether they believe the program worked (Blamey and McKenzie, 2007: 449; Mayne, 2012).”
BackgroundTheories of Change and Realistic Evaluation: Peas in a Pod or Apples and Oranges?
Blamey, A., & Mackenzie, M. (2007) · Evaluation · doi:10.1177/1356389007082129 · Cited by 383
Broadening the range of designs and methods for impact evaluations
Stern, E., Stame, N., Mayne, J., Forss, K., Davies, R., & Befani, B. (2012) · doi:10.22163/fteval.2012.100 · Cited by 369 · cited 4×
“Advocates of the counterfactual approach to causality have had a strong influence on the debate on evaluation designs and ‘succeeded in challenging all evaluators to address questions of causal claims and explaining the effectiveness of development interventions’ (Stern et al., 2012: 8).”
BackgroundInfluential
Cited by
Cairney, P., & Oliver, K. (2017) · Health Research Policy and Systems · doi:10.1186/s12961-017-0192-x · Cited by 526
Multi-Method Research in the Social Sciences: A Review of Recent Frameworks and a Way Forward
Beach, D. (2020) · Government and Opposition · doi:10.1017/gov.2018.53 · Cited by 43
Self-citationPallas, S. W., & Ruger, J. P. (2017) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.01.004 · Cited by 16
Metadata
- Authors
- Schmitt, J., & Beach, D.
- Venue
- Evaluation
- Publisher
- SAGE Publications
- Published
- 2015-10
- Volume
- 21
- Issue
- 4
- Pages
- 429-447
- ISSN
- 1356-3890 (ISSN-L)
- 1461-7153
- OpenAlex
- W2337550509
- Cited by
- 53
- References
- 16
- FWCI
- 13.29
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The contribution of process tracing to theory-based evaluations of complex aid instruments. https://app.cochat.ai/papers/10.1177/1356389015607739
- The paper
- Schmitt, J., & Beach, D. (2015). The contribution of process tracing to theory-based evaluations of complex aid instruments. Evaluation. https://doi.org/10.1177/1356389015607739
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.