What is mechanistic evidence, and why do we need it for evidence-based policy?
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
It has recently been argued that successful evidence-based policy should rely on two kinds of evidence: statistical and mechanistic. The former is held to be evidence that a policy brings about the desired outcome, and the latter concerns how it does so. Although agreeing with the spirit of this proposal, we argue that the underlying conception of mechanistic evidence as evidence that is different in kind from correlational, difference-making or statistical evidence, does not correctly capture the role that information about mechanisms should play in evidence-based policy. We offer an alternative account of mechanistic evidence as information concerning the causal pathway connecting the policy intervention to its outcome. Not only can this be analyzed as evidence of difference-making, it is also to be found at any level and is obtainable by a broad range of methods, both experimental and observational. Using behavioral policy as an illustration, we draw the implications of this revised understanding of mechanistic evidence for debates concerning policy extrapolation, evidence hierarchies, and evidence integration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841 · cited 1×
“Here again, as in the case of mechanism experiments, prior mechanistic knowledge and hypotheses about how the policy works are useful guides for identifying which variables for stratification would be most informative, for example (Deaton and Cartwright 2018).”
Interpreting Causality in the Health Sciences
Russo, F., & Williamson, J. (2007) · International Studies in the Philosophy of Science · doi:10.1080/02698590701498084 · Cited by 549
Mechanisms and the Evidence Hierarchy
Clarke, B., Gillies, D., Illari, P., Russo, F., & Williamson, J. (2014) · Topoi · doi:10.1007/s11245-013-9220-9 · Cited by 253
Selecting Appropriate Cases When Tracing Causal Mechanisms
Beach, D., & Pedersen, R. B. (2018) · Sociological Methods & Research · doi:10.1177/0049124115622510 · Cited by 198
Mechanistic Evidence: Disambiguating the Russo–Williamson Thesis
Illari, P. M. (2011) · International Studies in the Philosophy of Science · doi:10.1080/02698595.2011.574856 · Cited by 171 · cited 2×
“On the one hand, they approvingly quote Illari (2011), who argues for a separation between kinds of evidence and kinds of methods.”
Process Tracing and Causal Mechanisms
Waldner, D. (2012) · Oxford Handbooks Online · doi:10.1093/oxfordhb/9780195392753.013.0004 · Cited by 131
Johnson, E. J., & Goldstein, D. (2003) · Science · doi:10.1126/science.1091721 · Cited by 35
Cited by
Self-nudging and the citizen choice architect
REIJULA, S., & HERTWIG, R. (2022) · Behavioural Public Policy · doi:10.1017/bpp.2020.5 · Cited by 130
Self-citationMcCoy, L. G., Brenna, C. T. A., Chen, S. S., Vold, K., & Das, S. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.001 · Cited by 118
“Plausible mechanistic suggestions are critical in the development of new hypotheses [51] , and may be useful in extrapolating the results of a statistical finding to a new population [ 52 , 53 ].”
BackgroundLuthfi, A., & Janssen, M. (2019) · International Journal on Advanced Science, Engineering and Information Technology · doi:10.18517/ijaseit.9.3.8846 · Cited by 22
“The unique characteristic inherent in evidence-based policy-making is prioritizing evidence meta-analysis and randomized trials compared to other evidence-based methods [27].”
BackgroundMethod
Metadata
- Authors
- Marchionni, C., & Reijula, S.
- Venue
- Studies in History and Philosophy of Science Part A
- Publisher
- Elsevier BV
- Published
- 2019-02
- Volume
- 73
- Pages
- 54-63
- ISSN
- 0039-3681 (ISSN-L)
- OpenAlex
- W2887550975
- Cited by
- 54
- References
- 7
- FWCI
- 23.36
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: What is mechanistic evidence, and why do we need it for evidence-based policy? https://app.cochat.ai/papers/10.1016/j.shpsa.2018.08.003
- The paper
- Marchionni, C., & Reijula, S. (2019). What is mechanistic evidence, and why do we need it for evidence-based policy? Studies in History and Philosophy of Science Part A. https://doi.org/10.1016/j.shpsa.2018.08.003
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.