Combining the regression discontinuity design and propensity score‐based weighting to improve causal inference in program evaluation
Open accessAbstract
The regression discontinuity (RD) design is considered to be the closest to a randomized trial that can be applied in non‐experimental settings. The design relies on a cut‐off point on a continuous baseline variable to assign individuals to treatment. The individuals just to the right and left of the cut‐off are assumed to be exchangeable – as in a randomized trial. Any observed discontinuity in the relationship between the assignment variable and outcome is therefore considered evidence of a treatment effect. In this paper, we describe key advances in the RD design over the past decade and illustrate their implementation using data from a health management intervention. We then introduce the propensity score‐based weighting technique as a complement to the RD design to correct for imbalances in baseline characteristics between treated and non‐treated groups that may bias RD results. We find that the weighting strategy outperforms standard regression covariate adjustment in the present data. One clear advantage of the weighting technique over regression covariate adjustment is that we can directly inspect the degree to which balance was achieved. Because of its relative simplicity and tremendous utility, the RD design (either alone or combined with propensity score weighting adjustment) should be considered as an alternative approach to evaluate health management program effectiveness when using observational data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Comparing results obtained using different methods can be used to assess the dependence of findings on particular assumptions.
“Comparing results obtained using different methods can be used to assess the dependence of findings on particular assumptions (50).”
From Natural Experiments: An Overview of Methods, Approaches, and Contributions to Public Health Intervention Research (2017) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241 · see citing works
A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity
White, H. (1980) · Econometrica · doi:10.2307/1912934 · Cited by 25,861 · see citing works
(2002) · Cited by 13,211 · see citing works
Practical Nonparametric Statistics
Groggel, D. J., & Conover, W. J. (2000) · Technometrics · doi:10.2307/1271101 · Cited by 11,071 · see citing works
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581 · see citing works
Regression discontinuity designs in economics
Lee, D. S., & Lemieux, T. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.2.281 · Cited by 4,534 · see citing works
Economic Costs of Diabetes in the U.S. in 2002
American Diabetes Association (2003) · Diabetes Care · doi:10.2337/diacare.26.3.917 · Cited by 4,147 · see citing works
Regression discontinuity designs: A guide to practice
Imbens, G. W., & Lemieux, T. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.001 · Cited by 3,701 · see citing works
Manipulation of the Running Variable in the Regression Discontinuity Design: A Density Test,
McCrary, J. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.005 · Cited by 3,600 · see citing works
Local Polynomial Modelling and its Applications
Fan, J., & Gijbels, I. (1994) · Springer eBooks · doi:10.1007/978-1-4899-3150-4 · Cited by 2,897 · see citing works
Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design
Hahn, J., Todd, P., & Klaauw, W. (2001) · Econometrica · doi:10.1111/1468-0262.00183 · Cited by 2,679 · see citing works
Graphical Methods for Data Analysis.
Healy, M. J. R., Chambers, J. M., Cleveland, W. S., Kleiner, B., & Tukey, P. A. (1984) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981587 · Cited by 2,315 · see citing works
Optimal Bandwidth Choice for the Regression Discontinuity Estimator
Imbens, G. W., & Kalyanaraman, K. (2011) · The Review of Economic Studies · doi:10.1093/restud/rdr043 · Cited by 2,135 · see citing works
Graphical Methods for Data Analysis.
Fisher, N. I., Chambers, J. M., Cleveland, W. S., Kleiner, B., & Tukey, P. A. (1984) · Biometrics · doi:10.2307/2531418 · Cited by 1,982 · see citing works
Regression-discontinuity analysis: An alternative to the ex post facto experiment.
Thistlethwaite, D. L., & Campbell, D. T. (1960) · Journal of Educational Psychology · doi:10.1037/h0044319 · Cited by 1,469 · see citing works
Rosenbaum, P. R. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478441 · Cited by 998 · see citing works
8. Misunderstandings Between Experimentalists and Observationalists About Causal Inference
Imai, K., King, G., & Stuart, E. A. (2008) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2007.00527.x · Cited by 895 · see citing works
Local Polynomial Modelling and Its Applications: Monographs on Statistics and Applied Probability 66
Fan, J.-Q. (1996) · Routledge eBooks · doi:10.1201/9780203748725 · Cited by 782 · see citing works
The Impact of Nearly Universal Insurance Coverage on Health Care Utilization: Evidence from Medicare
Card, D., Dobkin, C., & Maestas, N. (2008) · American Economic Review · doi:10.1257/aer.98.5.2242 · Cited by 731 · see citing works
Optimal Bandwidth Choice for the Regression Discontinuity Estimator
Imbens, G. W., & Kalyanaraman, K. (2009) · doi:10.1920/wp.cem.2010.0510 · Cited by 439 · see citing works
Causal inference with observational data
Nichols, A. (2007) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0800700403 · Cited by 339 · see citing works
Standard Distance in Univariate and Multivariate Analysis
Flury, B. K., & Riedwyl, H. (1986) · The American Statistician · doi:10.1080/00031305.1986.10475403 · Cited by 289 · see citing works
On automatic boundary corrections
Cheng, M., Fan, J., & Marron, J. S. (1997) · The Annals of Statistics · doi:10.1214/aos/1031594737 · Cited by 274 · see citing works
From association to causation: some remarks on the history of statistics
Freedman, D. A. (1999) · Statistical Science · doi:10.1214/ss/1009212409 · Cited by 243 · see citing works
Research Design for Program Evaluation: The Regression-Discontinuity Approach.
Spiegelman, C. H., & Trochim, W. M. K. (1986) · Journal of the American Statistical Association · doi:10.2307/2289044 · Cited by 204 · see citing works
Research design for program evaluation: The regression-discontinuity approach
Spiegelman, C. H. (1985) · Evaluation and Program Planning · doi:10.1016/0149-7189(85)90035-7 · Cited by 190 · see citing works
Rosenblum, M. (2011) · Springer series in statistics · doi:10.1007/978-1-4419-9782-1_9 · Cited by 185 · see citing works
Research methodology: Strengthening causal interpretations of nonexperimental data
(1990) · Cancer Causes & Control · doi:10.1007/bf00053175 · Cited by 113 · see citing works
Standard Distance in Univariate and Multivariate Analysis
Flury, B. K., & Riedwyl, H. (1986) · The American Statistician · doi:10.2307/2684560 · Cited by 77 · see citing works
Causal Inference with Observational Data
Nichols, A. (2007) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0700700403 · Cited by 73 · see citing works
Linden, A., & Adams, J. (2010) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2009.01219.x · Cited by 65 · see citing works
Self-citationLinden, A., Adams, J., & Roberts, N. E. (2003) · Disease Management · doi:10.1089/109350703321908478 · Cited by 57 · see citing works
Self-citationBuckley, J., & Shang, Y. (2020) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.7275/dwxt-zv80 · Cited by 54 · see citing works
Linden, A., Adams, J., & Roberts, N. E. (2006) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2005.00573.x · Cited by 54 · see citing works
Self-citationErratum and Discussion of Propensity-Score Reweighting
Nichols, A. (2008) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0800800405 · Cited by 52 · see citing works
Evaluating Disease Management Program Effectiveness
Linden, A., Adams, J., & Roberts, N. E. (2005) · Disease Management & Health Outcomes · doi:10.2165/00115677-200513030-00002 · Cited by 40 · see citing works
Self-citationTrochim, W. M. K., Cappelleri, J. C., & Reichardt, C. S. (1991) · Evaluation Review · doi:10.1177/0193841x9101500504 · Cited by 31 · see citing works
Identifying spin in health management evaluations
Linden, A. (2011) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2010.01611.x · Cited by 15 · see citing works
Self-citationLinden, A., Biuso, T. J., Allada, G., Barker, A. F., Cigarroa, J. E., Haranath, S. P., Rinkevich, D., & Stajduhar, K. C. (2007) · Disease Management & Health Outcomes · doi:10.2165/00115677-200715050-00006 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- American Diabetes Association (2003) Economic costs of diabetes in the US in 2002. Diabetes Care, 26 (3), 917-932.
Cited by
Sorted by citations
Craig, P., Katikireddi, S. V., Leyland, A., & Popham, F. (2017) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044327 · Cited by 570 · see citing works
Kitchen, J. A., Sonnert, G., & Sadler, P. M. (2018) · Science Education · doi:10.1002/sce.21332 · Cited by 203 · see citing works
Assessing regression to the mean effects in health care initiatives
Linden, A. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-119 · Cited by 195 · see citing works
Self-citationO’Keeffe, A. G., Geneletti, S., Baio, G., Sharples, L. D., Nazareth, I., & Petersen, I. (2014) · BMJ · doi:10.1136/bmj.g5293 · Cited by 119 · see citing works
Oversight and Efficiency in Public Projects: A Regression Discontinuity Analysis
Calvo, E., Cui, R., & Serpa, J. C. (2016) · Management Science · doi:10.1287/mnsc.2018.3202 · Cited by 75 · see citing works
Multilevel Interventions Targeting Obesity: Research Recommendations for Vulnerable Populations
Stevens, J., PRATT, C. A., Boyington, J., Nelson, C. R., Truesdale, K. P., Ward, D. S., Lytle, L. A., Sherwood, N. E., Robinson, T. N., Moore, S. M., et al. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.09.011 · Cited by 70 · see citing works
Nikaj, S., & Lund, P. K. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002543 · Cited by 67 · see citing works
Alternatives to Randomized Control Trial Designs for Community-Based Prevention Evaluation
Henry, D. B., Tolan, P. H., Gorman‐Smith, D., & Schoeny, M. E. (2016) · Prevention Science · doi:10.1007/s11121-016-0706-8 · Cited by 42 · see citing works
Wallace, J., & Song, Z. (2016) · Health Affairs · doi:10.1377/hlthaff.2015.1195 · Cited by 35 · see citing works
Villamizar‐Villegas, M., Pinzon‐Puerto, F. A., & Ruiz-Sánchez, M. A. (2021) · Journal of Economic Surveys · doi:10.1111/joes.12461 · Cited by 34 · see citing works
Linden, A., & Yarnold, P. R. (2016) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12610 · Cited by 31 · see citing works
Self-citationUsing machine learning to model dose–response relationships
Linden, A., Yarnold, P. R., & Nallamothu, B. K. (2016) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12573 · Cited by 30 · see citing works
Self-citationGuo, C., Pang, L., Wen, X., & Zheng, X. (2018) · Journal of Women s Health · doi:10.1089/jwh.2018.7097 · Cited by 29 · see citing works
Using classification tree analysis to generate propensity score weights
Linden, A., & Yarnold, P. R. (2017) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12744 · Cited by 26 · see citing works
Self-citationUsing machine learning to assess covariate balance in matching studies
Linden, A., & Yarnold, P. R. (2016) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12538 · Cited by 25 · see citing works
Self-citationKitchen, J. A., Chen, C., Sonnert, G., & Sadler, P. M. (2022) · Teachers College Record The Voice of Scholarship in Education · doi:10.1177/01614681221086445 · Cited by 25 · see citing works
Effect of input credit on smallholder farmers’ output and income
Iddrisu, A., Ansah, I. G. K., & Nkegbe, P. K. (2017) · Agricultural Finance Review · doi:10.1108/afr-05-2017-0032 · Cited by 23 · see citing works
A comparison of approaches for stratifying on the propensity score to reduce bias
Linden, A. (2017) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12701 · Cited by 21 · see citing works
Self-citationCombining machine learning and matching techniques to improve causal inference in program evaluation
Linden, A., & Yarnold, P. R. (2016) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12592 · Cited by 21 · see citing works
Self-citationRuan, H., Ran, X., Li, S., & Zhang, Q. (2024) · Lipids in Health and Disease · doi:10.1186/s12944-024-02061-9 · Cited by 20 · see citing works
Ouya, F. O., Murage, A. W., Pittchar, J. O., Chidawanyika, F., Pickett, J. A., & Khan, Z. R. (2023) · Agriculture & Food Security · doi:10.1186/s40066-023-00418-4 · Cited by 17 · see citing works
Is Cigarette and Alcohol Use Associated with High-Risk Sexual Behaviors Among Youth in China?
Guo, C., Wen, X., Li, N., Wang, Z., Chen, G., & Zheng, X. (2017) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2017.03.249 · Cited by 15 · see citing works
Zhang, K.-J., Liang, L., Diao, Y., Xie, Y.-M., Wang, D.-D., Xu, F., Ye, T., Lu, W., Cheng, J., Shen, G., et al. (2022) · Surgery Today · doi:10.1007/s00595-022-02576-7 · Cited by 16 · see citing works
Beumer, B. R., de Wilde, R. F., Metselaar, H. J., de Man, R. A., Polak, W. G., & IJzermans, J. N. M. (2021) · Cancers · doi:10.3390/cancers13153730 · Cited by 15 · see citing works
Linden, A. (2016) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12668 · Cited by 14 · see citing works
Self-citationMitigating information frictions in trade: Evidence from export credit guarantees
Agarwal, N., Chan, J. M. L., Lodefalk, M., Tang, A., Tano, S., & Wang, Z. (2023) · Journal of International Economics · doi:10.1016/j.jinteco.2023.103831 · Cited by 13 · see citing works
Linden, A. (2016) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12625 · Cited by 13 · see citing works
Self-citationKitchen, J. A., Williams, M. S., Sonnert, G., & Sadler, P. M. (2023) · International Journal of Science Education · doi:10.1080/09500693.2023.2220071 · Cited by 12 · see citing works
THE IMPACT OF LOCAL WAGE REGULATION ON EMPLOYMENT: A BORDER ANALYSIS FROM ITALY IN THE 1950s*
de Blasio, G., & Poy, S. (2014) · Journal of Regional Science · doi:10.1111/jors.12301 · Cited by 136 · see citing works
Estimating causal effects with optimization-based methods: A review and empirical comparison
Cousineau, M., Verter, V., Murphy, S. A., & Pineau, J. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2022.01.046 · Cited by 11 · see citing works
Andersen, O. W., Basile, I., de Kemp, A., Gotz, G., Lundsgaarde, E., & Orth, M. (2019) · OECD development co-operation working papers · doi:10.1787/4c1fc76e-en · Cited by 11 · see citing works
Rasella, D., Morais, G. A. D. S., Anderle, R. V. R., da Silva, A. F., Lua, I., Coelho, R. D. A., Rubio, F. A., Magno, L., Machado, D. B., Pescarini, J. M., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0265253 · Cited by 10 · see citing works
Campus Visits: Impact of a College Outreach Strategy on Student STEM Aspirations
Kitchen, J. A., Sonnert, G., & Sadler, P. M. (2019) · NASPA Journal · doi:10.1080/19496591.2019.1653312 · Cited by 10 · see citing works
Freedom House’s Scarlet Letter
Roberts, J. A., & Tellez, J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108763493.009 · Cited by 8 · see citing works
Dirichlet process mixture models for regression discontinuity designs
Ricciardi, F., Liverani, S., & Baio, G. (2022) · Statistical Methods in Medical Research · doi:10.1177/09622802221129044 · Cited by 7 · see citing works
Regression discontinuity designs in education: a practitioner’s guide
Suk, Y. (2024) · Asia Pacific Education Review · doi:10.1007/s12564-024-09956-3 · Cited by 5 · see citing works
Political Economic Determinants of Growth Acceleration: A Korea-Indonesia Comparative Study
Wardhana, I. W. (2016) · Kajian Ekonomi dan Keuangan · doi:10.31685/kek.v20i1.182 · Cited by 5 · see citing works
Propensity score modeling for business marketing research
Guenther, P., Guenther, M., Misra, S., Koval, M., & Zaefarian, G. (2025) · Industrial Marketing Management · doi:10.1016/j.indmarman.2025.03.006 · Cited by 3 · see citing works
Regression Discontinuity Designs: A Decision Theoretic ApproachPreprint
Constantinou, P., & O’Keeffe, A. G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.00439 · Cited by 2 · see citing works
Prognostic value of TC/HDL-C ratios in ischemic stroke: a multicenter observational study
Ruan, H., Zhang, Q., Zhang, P., Tang, Y., & Ran, X. (2026) · Brain Hemorrhages · doi:10.1016/j.hest.2026.01.003 · Cited by 1 · see citing works
Shobo, Y., Wong, J. D., & Bell, A. (2014) · Society for Research on Educational Effectiveness
Implementation Science: Identifying the Path from Innovation to Impact
Madon, T. (2015) · Sage eBooks · doi:10.4135/9781483398839.n23
Essays in Tax Policy Evaluation
Hamilton, S. (2018) · Deep Blue (University of Michigan)
Towards new frontiers: How attainment discrepancy affects exploratory behavior in crowdfunding
Ilk, N., Shang, G., Cui, Z., & Zhang, Z. T. (2024) · Information & Management · doi:10.1016/j.im.2024.104053
Ruan, H., Ran, X., Li, S.-S., & Zhang, Q. (2024) · Figshare · doi:10.6084/m9.figshare.26691277
Ruan, H., Ran, X., Li, S.-S., & Zhang, Q. (2024) · Figshare · doi:10.6084/m9.figshare.25376169
Ruan, H., Ran, X., Li, S.-S., & Zhang, Q. (2024) · Figshare · doi:10.6084/m9.figshare.25376141
Ruan, H., Ran, X., Li, S.-S., & Zhang, Q. (2024) · Figshare · doi:10.6084/m9.figshare.26691283
Ruan, H., Ran, X., Li, S.-S., & Zhang, Q. (2024) · Figshare · doi:10.6084/m9.figshare.26691289
Ruan, H., Ran, X., Li, S.-S., & Zhang, Q. (2024) · Figshare · doi:10.6084/m9.figshare.26691280
- Authors
- Linden, A., & Adams, J. L.
- Affiliations
- University of Michigan (US)
- RAND Corporation (US)
- Corresponding author
- Linden, Ariel
- Venue
- Journal of Evaluation in Clinical Practice
- Publisher
- Wiley
- Published
- 2012-02-05
- Volume
- 18
- Issue
- 2
- Pages
- 317-325
- ISSN
- 1356-1294 (ISSN-L)
- 1365-2753
- OpenAlex
- W1910000333
- Cited by
- 50
- References
- 38
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 9 · 2023: 3 · 2022: 5 · 2021: 2 · 2020: 1 · 2019: 2 · 2018: 4 · 2017: 5 · 2016: 13 · 2015: 1 · 2014: 2 · 2013: 1
- FWCI
- 1.22
- Type
- journal article
- Language
- en
- Open access
- green
- pareonline.net
- Locations
- Journal of Evaluation in Clinical Practice (publishedVersion)
- PubMed (publishedVersion)
- Deep Blue (University of Michigan) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- regression discontinuity design, propensity score weighting, causal inference, program evaluation, observational data, treatment effect estimation
- MeSH terms
- Female, Health Services Research, Humans, Male, Regression Analysis, Research Design, Reproducibility of Results, Program Evaluation, Outcome Assessment, Health Care, Risk Assessment, Patient-Centered Care, Disease Management, Propensity Score
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
