The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units
Open accessAbstract
This paper examines the synthetic control method in contrast to commonly used difference‐in‐differences (DiD) estimation, in the context of a re‐evaluation of a pay‐for‐performance (P4P) initiative, the Advancing Quality scheme. The synthetic control method aims to estimate treatment effects by constructing a weighted combination of control units, which represents what the treated group would have experienced in the absence of receiving the treatment. While DiD estimation assumes that the effects of unobserved confounders are constant over time, the synthetic control method allows for these effects to change over time, by re‐weighting the control group so that it has similar pre‐intervention characteristics to the treated group. We extend the synthetic control approach to a setting of evaluation of a health policy where there are multiple treated units. We re‐analyse a recent study evaluating the effects of a hospital P4P scheme on risk‐adjusted hospital mortality. In contrast to the original DiD analysis, the synthetic control method reports that, for the incentivised conditions, the P4P scheme did not significantly reduce mortality and that there is a statistically significant increase in mortality for non‐incentivised conditions. This result was robust to alternative specifications of the synthetic control method. © 2015 The Authors. Health Economics published by John Wiley & Sons Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deci, E. L., Koestner, R., & Ryan, R. M. (1999) · Psychological Bulletin · doi:10.1037/0033-2909.125.6.627 · Cited by 7,176
Abadie, A., Diamond, A., & Hainmueller, J. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.ap08746 · Cited by 5,513
The economic costs of conflict: a case study of the Basque Country
Abadie, A., & Gardeazabal, J. (2003) · American Economic Review · doi:10.1257/000282803321455188 · Cited by 4,517
Robust Inference With Multiway Clustering
Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2011) · Journal of Business & Economic Statistics · doi:10.1198/jbes.2010.07136 · Cited by 3,203
Comparative Politics and the Synthetic Control Method
Abadie, A., Diamond, A., & Hainmueller, J. (2015) · American Journal of Political Science · doi:10.1111/ajps.12116 · Cited by 2,406
Semiparametric Difference-in-Differences Estimators
Abadie, A. (2005) · The Review of Economic Studies · doi:10.1111/0034-6527.00321 · Cited by 2,286
Edgington, E., Edgington, E., & Onghena, P. (2007) · doi:10.1201/9781420011814 · Cited by 2,030
Edgington, E. S. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess2169 · Cited by 2,028
Sekhon, J. S. (2011) · Journal of Statistical Software · doi:10.18637/jss.v042.i07 · Cited by 1,543
Estimates of Location Based on Rank Tests
Hodges, J. L., & Lehmann, E. L. (1963) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704172 · Cited by 1,260
Identification and Inference in Nonlinear Difference-in-Differences Models
Athey, S., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00668.x · Cited by 1,084
Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models
Xu, Y. (2017) · Political Analysis · doi:10.1017/pan.2016.2 · Cited by 938
Craig, P., Cooper, C., Gunnell, D., Haw, S., Lawson, K., Macintyre, S., Ogilvie, D., Petticrew, M., Reeves, B., Sutton, M., et al. (2012) · Journal of Epidemiology and Community Health · doi:10.1136/jech-2011-200375 · Cited by 912
Self-citationAlternative approaches to evaluation in empirical microeconomics
Blundell, R., & Costa Dias, M. (2008) · doi:10.1920/wp.cem.2008.2608 · Cited by 881
Quality Disclosure and Certification: Theory and Practice
Dranove, D., & Jin, G. Z. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.4.935 · Cited by 740
Effects of Pay for Performance on the Quality of Primary Care in England
Campbell, S. M., Reeves, D., Kontopantelis, E., Sibbald, B., & Roland, M. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0807651 · Cited by 696
Assessing Economic Liberalization Episodes: A Synthetic Control Approach
Billmeier, A., & Nannicini, T. (2013) · Review of Economics and Statistics · doi:10.1162/rest_a_00324 · Cited by 506
Quality of Care in U.S. Hospitals as Reflected by Standardized Measures, 2002–2004
Williams, S. C., Schmaltz, S. P., Morton, D. J., Koss, R. G., & Loeb, J. M. (2005) · New England Journal of Medicine · doi:10.1056/nejmsa043778 · Cited by 421
How Much Should We Trust Differences-in-Differences Estimates?
Bertrand, M., Duflo, E., & Mullainathan, S. (2002) · National Bureau of Economic Research · doi:10.3386/w8841 · Cited by 409
Alternative Approaches to Evaluation in Empirical Microeconomics
Blundell, R., & Costa Dias, M. (2009) · Journal of Human Resources · doi:10.1353/jhr.2009.0009 · Cited by 394
Why We Should Not Be Indifferent to Specification Choices for Difference‐in‐Differences
Ryan, A. M., Burgess, J. F., & Dimick, J. B. (2015) · Health Services Research · doi:10.1111/1475-6773.12270 · Cited by 320
Hinrichs, P. (2012) · Review of Economics and Statistics · doi:10.1162/rest_a_00170 · Cited by 283
Schaffer, M. E. (2012) · RePEc: Research Papers in Economics · Cited by 263
Doran, T., Kontopantelis, E., Valderas, J. M., Campbell, S., Roland, M., Salisbury, C., & Reeves, D. (2011) · BMJ · doi:10.1136/bmj.d3590 · Cited by 258
The Value of Connections in Turbulent Times: Evidence from the United States
Acemoglu, D., Johnson, S., Kermani, A., Kwak, J., & Mitton, T. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2363609 · Cited by 232
Teacher Incentives and Student Achievement: Evidence from New York City Public Schools
Fryer, R. (2011) · National Bureau of Economic Research · doi:10.3386/w16850 · Cited by 217
Coffman, M., & Noy, I. (2012) · Environment and Development Economics · doi:10.1017/s1355770x11000350 · Cited by 213
Reduced Mortality with Hospital Pay for Performance in England
Sutton, M., Nikolova, S., Boaden, R., Lester, H., McDonald, R., & Roland, M. (2012) · New England Journal of Medicine · doi:10.1056/nejmsa1114951 · Cited by 207
Self-citationDoes Universal Coverage Improve Health? The Massachusetts Experience
Courtemanche, C. J., & Zapata, D. (2014) · Journal of Policy Analysis and Management · doi:10.1002/pam.21737 · Cited by 190
Key issues in the design of pay for performance programs
Eijkenaar, F. (2013) · The European Journal of Health Economics · doi:10.1007/s10198-011-0347-6 · Cited by 180
Doctor Behaviour under a Pay for Performance Contract: Treating, Cheating and Case Finding?
Gravelle, H., Sutton, M., & Ma, A. (2010) · The Economic Journal · doi:10.1111/j.1468-0297.2009.02340.x · Cited by 166
Self-citationEconomic evaluation of pay-for-performance in health care: a systematic review
Emmert, M., Eijkenaar, F., Kemter, H., Esslinger, A. S., & Schöffski, O. (2011) · The European Journal of Health Economics · doi:10.1007/s10198-011-0329-8 · Cited by 139
Ryan, A. M. (2009) · Health Services Research · doi:10.1111/j.1475-6773.2009.00956.x · Cited by 134
Long-Term Effect of Hospital Pay for Performance on Mortality in England
Kristensen, S. R., Meacock, R., Turner, A. J., Boaden, R., McDonald, R., Roland, M. O., & Sutton, M. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1400962 · Cited by 119
Self-citationDo stimulant medications improve educational and behavioral outcomes for children with ADHD?
Currie, J., Stabile, M., & Jones, L. E. (2014) · Journal of Health Economics · doi:10.1016/j.jhealeco.2014.05.002 · Cited by 112
Why pay for performance may be incompatible with quality improvement
Woolhandler, S. J., Dan Ariely, C., & Himmelstein, D. U. (2012) · BMJ · doi:10.1136/bmj.e5015 · Cited by 70
Meacock, R., Kristensen, S. R., & Sutton, M. (2013) · Health Economics · doi:10.1002/hec.2978 · Cited by 56
Self-citationEconometric Evaluation of Health Policies
Jones, A. M., & Rice, N. (2011) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199238828.013.0037 · Cited by 54
The Value of Connections in Turbulent Times: Evidence from the United States
Acemoğlu, D., Johnson, S., Kermani, A., Kwak, J., & Mitton, T. (2013) · National Bureau of Economic Research · doi:10.3386/w19701 · Cited by 43
Reduced Mortality With Hospital Pay for Performance in England
Sutton, M., Nikolova, S., Boaden, R., Lester, H. E., McDonald, R., & Roland, M. O. (2013) · Obstetrical & Gynecological Survey · doi:10.1097/01.ogx.0000428158.81933.54 · Cited by 34
Self-citationExact Confidence Intervals for Nonconstant Effects by Inverting the Signed Rank Test
Rosenbaum, P. R. (2003) · The American Statistician · doi:10.1198/0003130031405 · Cited by 32
Échevin, D., & Fortin, B. (2014) · Journal of Health Economics · doi:10.1016/j.jhealeco.2014.03.008 · Cited by 25
Inference with Difference-in-Differences RevisitedPreprint
Brewer, M., Crossley, T. F., & Joyce, R. C. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2363229 · Cited by 25
Lagarde, M., Wright, M., Nossiter, J., & Mays, N. (2013) · LSHTM Research Online (London School of Hygiene and Tropical Medicine) · Cited by 20
Dubé, A., & Zipperer, B. (2013) · Cited by 9
An empirical investigation of why doctors migrate and women fail to go for screening
Okeke, E. N. (2009) · Deep Blue (University of Michigan) · Cited by 6
Cited by
Designing Difference in Difference Studies: Best Practices for Public Health Policy Research
Wing, C., Simon, K., & Bello-Gomez, R. A. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-013507 · Cited by 1,470
Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects
Abadie, A. (2021) · Journal of Economic Literature · doi:10.1257/jel.20191450 · Cited by 1,486
Craig, P., Katikireddi, S. V., Leyland, A., & Popham, F. (2017) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044327 · Cited by 570
Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., et al. (2021) · Health Technology Assessment · doi:10.3310/hta25570 · Cited by 522
Synthetic controls with staggered adoption.
Ben-Michael, E., Feller, A., & Rothstein, J. (2022) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12448 · Cited by 120
Metadata
- Authors
- Kreif, N., Grieve, R., Hangartner, D., Turner, A. J., Nikolova, S., & Sutton, M.
- Venue
- Health Economics
- Publisher
- Wiley
- Published
- 2015-10-07
- Volume
- 25
- Issue
- 12
- Pages
- 1514-1528
- ISSN
- 1057-9230 (ISSN-L)
- 1099-1050
- OpenAlex
- W2114492847
- Cited by
- 301
- References
- 46
- FWCI
- 14.02
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.