Robust nonparametric confidence intervals for regression-discontinuity designs
Open accessAbstract
In the regression-discontinuity (RD) design, units are assigned to treatment based on whether their value of an observed covariate exceeds a known cutoff. In this design, local polynomial estimators are now routinely employed to construct confidence intervals for treatment effects. The performance of these confidence intervals in applications, however, may be seriously hampered by their sensitivity to the specific bandwidth employed. Available bandwidth selectors typically yield a “large” bandwidth, leading to data-driven confidence intervals that may be biased, with empirical coverage well below their nominal target. We propose new theory-based, more robust confidence interval estimators for average treatment effects at the cutoff in sharp RD, sharp kink RD, fuzzy RD, and fuzzy kink RD designs. Our proposed confidence intervals are constructed using a bias-corrected RD estimator together with a novel standard error estimator. For practical implementation, we discuss mean squared error optimal bandwidths, which are by construction not valid for conventional confidence intervals but are valid with our robust approach, and consistent standard error estimators based on our new variance formulas. In a special case of practical interest, our procedure amounts to running a quadratic instead of a linear local regression. More generally, our results give a formal justification to simple inference procedures based on increasing the order of the local polynomial estimator employed. We find in a simulation study that our confidence intervals exhibit close-to-correct empirical coverage and good empirical interval length on average, remarkably improving upon the alternatives available in the literature. All results are readily available in R and STATA using our companion software packages described in Calonico, Cattaneo, and Titiunik (2014d, 2014b).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Selection of the bandwidth was conducted using the data-driven mean-squared error-reducing strategy proposed by Calonico, Cattaneo, and Titiunik (2014).
“Selection of 𝑧𝑧 𝑖𝑖 was conducted using the data-driven mean-squared error-reducing strategy, proposed by Calonico, Cattaneo, and Titiunik (2014 ).”
From More is Better: Evaluating the Impact of a Variation in Cash Assistance on the Reintegration Outcomes of Returning Afghan Refugees (2022) · cites this paper as Calonico, Cattaneo, and Titiunik (2014)
The regression discontinuity estimator assumes that the variable determining exposure is continuous, but it has also been used when that variable is discrete but takes many values.
“The RD estimator in Calonico et al [19] assumes that the variable determining exposure is continuous, but it has also been used when that variable is discrete but takes a large number of values.”
From Bacille Calmette-Guérin Vaccination in Infancy Does Not Protect Against Coronavirus Disease 2019 (COVID-19): Evidence From a Natural Experiment in Sweden (2021) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Regression discontinuity designs in economics
Lee, D. S., & Lemieux, T. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.2.281 · Cited by 4,534 · see citing works
Regression discontinuity designs: A guide to practice
Imbens, G. W., & Lemieux, T. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.001 · Cited by 3,701 · see citing works
Manipulation of the Running Variable in the Regression Discontinuity Design: A Density Test,
McCrary, J. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.005 · Cited by 3,600 · see citing works
Large sample properties of matching estimators for average treatment effects
Abadie, A., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00655.x · Cited by 2,908 · see citing works
Local Polynomial Modelling and its Applications
Fan, J., & Gijbels, I. (1994) · Springer eBooks · doi:10.1007/978-1-4899-3150-4 · Cited by 2,897 · see citing works
Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design
Hahn, J., Todd, P., & Klaauw, W. (2001) · Econometrica · doi:10.1111/1468-0262.00183 · Cited by 2,679 · see citing works
WAND, M. P., & Jones, M. C. (1995) · Springer eBooks · doi:10.1007/978-1-4899-4493-1 · Cited by 2,676 · see citing works
Optimal Bandwidth Choice for the Regression Discontinuity Estimator
Imbens, G. W., & Kalyanaraman, K. (2011) · The Review of Economic Studies · doi:10.1093/restud/rdr043 · Cited by 2,135 · see citing works
Randomized experiments from non-random selection in U.S. House elections
Lee, D. S. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.004 · Cited by 1,732 · see citing works
Regression-discontinuity analysis: An alternative to the ex post facto experiment.
Thistlethwaite, D. L., & Campbell, D. T. (1960) · Journal of Educational Psychology · doi:10.1037/h0044319 · Cited by 1,469 · see citing works
Local Polynomial Modelling and its Applications
Copeland, K. A. F. (1997) · Journal of Quality Technology · doi:10.1080/00224065.1997.11979762 · Cited by 1,203 · see citing works
Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity Design
Ludwig, J., & Miller, D. L. (2007) · The Quarterly Journal of Economics · doi:10.1162/qjec.122.1.159 · Cited by 978 · see citing works
Regression discontinuity inference with specification error
Lee, D. S., & Card, D. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.003 · Cited by 855 · see citing works
Robust Data-Driven Inference in the Regression-Discontinuity Design
Calónico, S., Cattaneo, M. D., & Titiunik, R. (2014) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x1401400413 · Cited by 735 · see citing works
Self-citationLi, K.-C. (1987) · The Annals of Statistics · doi:10.1214/aos/1176350486 · Cited by 510 · see citing works
Optimal Data-Driven Regression Discontinuity Plots
Calónico, S., Cattaneo, M. D., & Titiunik, R. (2015) · Journal of the American Statistical Association · doi:10.1080/01621459.2015.1017578 · Cited by 469 · see citing works
Self-citationHorowitz, J. L. (2001) · Handbook of econometrics · doi:10.1016/s1573-4412(01)05005-x · Cited by 407 · see citing works
Attanasio, O. P., Meghir, C., & Santiago, A. (2011) · The Review of Economic Studies · doi:10.1093/restud/rdr015 · Cited by 405 · see citing works
Geographic Boundaries as Regression Discontinuities
Keele, L. J., & Titiunik, R. (2015) · Political Analysis · doi:10.1093/pan/mpu014 · Cited by 391 · see citing works
Self-citationCattaneo, M. D., Frandsen, B., & Titiunik, R. (2014) · Journal of Causal Inference · doi:10.1515/jci-2013-0010 · Cited by 350 · see citing works
Self-citationEstimation in the Regression Discontinuity Model
Porter, J. (2003) · Cited by 285 · see citing works
On automatic boundary corrections
Cheng, M., Fan, J., & Marron, J. S. (1997) · The Annals of Statistics · doi:10.1214/aos/1031594737 · Cited by 274 · see citing works
Inference on Causal Effects in a Generalized Regression Kink Design
Card, D., Lee, D. S., Pei, Z., & Weber, A. M. (2015) · Econometrica · doi:10.3982/ecta11224 · Cited by 269 · see citing works
Regression–Discontinuity Analysis: A Survey of Recent Developments in Economics
van der Klaauw, W. (2008) · Labour · doi:10.1111/j.1467-9914.2008.00419.x · Cited by 242 · see citing works
rdrobust: An R Package for Robust Nonparametric Inference in Regression-Discontinuity Designs
Calónico, S., Cattaneo, M. D., & Titiunik, R. (2015) · The R Journal · doi:10.32614/rj-2015-004 · Cited by 232 · see citing works
Self-citationProgram Evaluation and Research Designs
DiNardo, J., & Lee, D. (2010) · Handbook of labour economics · doi:10.1016/s0169-7218(11)00411-4 · Cited by 146 · see citing works
Quantile treatment effects in the regression discontinuity design
Frandsen, B., Frölich, M., & Melly, B. (2012) · Journal of Econometrics · doi:10.1016/j.jeconom.2012.02.004 · Cited by 139 · see citing works
Pei, Z., Card, D., Lee, D. S., & Weber, A. M. (n.d.) · RePEc: Research Papers in Economics · Cited by 89 · see citing works
Cattaneo, M. D., & Farrell, M. H. (2013) · Journal of Econometrics · doi:10.1016/j.jeconom.2013.02.002 · Cited by 72 · see citing works
Self-citationWeak Identification in Fuzzy Regression Discontinuity Designs
Feir, D. L., Lemieux, T., & Marmer, V. (2015) · Journal of Business and Economic Statistics · doi:10.1080/07350015.2015.1024836 · Cited by 69 · see citing works
Chapter 74 Implementing Nonparametric and Semiparametric Estimators
Ichimura, H., & Todd, P. (2007) · Handbook of econometrics · doi:10.1016/s1573-4412(07)06074-6 · Cited by 56 · see citing works
Estimation of the Conditional Variance in Paired Experiments
Abadie, A., & Imbens, G. W. (2007) · Cited by 41 · see citing works
Estimation of the Conditional Variance in Paired Experiments
ABADIE, & IMBENS (2008) · Annales d Économie et de Statistique · doi:10.2307/27917244 · Cited by 38 · see citing works
Inference on Causal Effects in a Generalized Regression Kink Design
Card, D., Lee, D. S., Pei, Z., & Weber, A. M. (2015) · doi:10.17848/wp15-218 · Cited by 38 · see citing works
Empirical likelihood for regression discontinuity design
Otsu, T., Xu, K., & Matsushita, Y. (2014) · Journal of Econometrics · doi:10.1016/j.jeconom.2014.04.023 · Cited by 37 · see citing works
Porter, J. (2003) · SSRN Electronic Journal · doi:10.2139/ssrn.373081 · Cited by 33 · see citing works
Jumpy or Kinky? Regression Discontinuity without the DiscontinuityPreprint
Dong, Y. (2010) · RePEc: Research Papers in Economics · Cited by 20 · see citing works
Empirical Likelihood for Regression Discontinuity DesignPreprint
Otsu, T., & Xu, K. (2011) · SSRN Electronic Journal · doi:10.2139/ssrn.1838350 · Cited by 14 · see citing works
Regression Discontinuity Marginal Threshold Treatment EffectsPreprint
Dong, Y., & Lewbel, A. (2010) · RePEc: Research Papers in Economics · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- FRANDSEN, B., M. FRÖLICH, AND B. MELLY (2012): "Quantile Treatments Effects in the Regres- sion Discontinuity Design," Journal of Econometrics, 168 (2), 382-395.
- ICHIMURA, H., AND P. E. TODD (2007): "Implementing Nonparametric and Semiparametric Es- timators," in Handbook of Econometrics, Vol. 6B, ed. by J. J. Heckman and E. E. Leamer. New York: Elsevier Science B.V., 5369-5468. [2298,2301]
Cited by
Sorted by citations
Do Investors Value Sustainability? A Natural Experiment Examining Ranking and Fund Flows
HARTZMARK, S. M., & SUSSMAN, A. B. (2019) · The Journal of Finance · doi:10.1111/jofi.12841 · Cited by 1,607 · see citing works
Affirmative Action, Mismatch, and Economic Mobility after California’s Proposition 209
Bleemer, Z. (2021) · The Quarterly Journal of Economics · doi:10.1093/qje/qjab027 · Cited by 149 · see citing works
The Long-Run Impacts of Financial Aid: Evidence from California’s Cal Grant
Bettinger, E., Gurantz, O., Kawano, L., Sacerdote, B., & Stevens, M. (2019) · American Economic Journal: Economic Policy · doi:10.1257/pol.20170466 · Cited by 110 · see citing works
Schudde, L., & Scott-Clayton, J. (2016) · Research in Higher Education · doi:10.1007/s11162-016-9413-3 · Cited by 87 · see citing works
Snapping Back: Food Stamp Bans and Criminal Recidivism
Tuttle, C. (2019) · American Economic Journal: Economic Policy · doi:10.1257/pol.20170490 · Cited by 85 · see citing works
Marx, B. M., & Turner, L. J. (2018) · American Economic Journal: Applied Economics · doi:10.1257/app.20160127 · Cited by 69 · see citing works
Gallo, R. J., Shieh, L., Smith, M., Marafino, B. J., Geldsetzer, P., Asch, S. M., Shum, K., Lin, S., Westphal, J., Hong, G., et al. (2024) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2024.0084 · Cited by 58 · see citing works
de Chaisemartin, C., & de Chaisemartin, L. (2021) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa1223 · Cited by 43 · see citing works
Algorithmic Risk Assessment in the Hands of Humans
Stevenson, M., & Doleac, J. L. (2020) · doi:10.2139/ssrn.3513695 · Cited by 42 · see citing works
Does Cash Bail Deter Misconduct?
Ouss, A., & Stevenson, M. (2023) · American Economic Journal: Applied Economics · doi:10.1257/app.20210349 · Cited by 28 · see citing works
Esper, H., Krishnan, N., & Wieser, C. (2022) · Policy Research Working Papers · doi:10.1596/1813-9450-9897
- Authors
- Calonico, S., Cattaneo, M. D., & Titiunik, R.
- Affiliations
- Dept. of Economics; University of Miami; 5250 University Dr. Coral Gables FL 33124 U.S.A.
- University of Michigan (US)
- Dept. of Political Science; University of Michigan; 505 S. State St. Ann Arbor MI 48109 U.S.A.
- Corresponding author
- Cattaneo, Matias D.
- ORCID
- Cattaneo, Matias D.
- Venue
- Econometrica
- Publisher
- The Econometric Society
- Published
- 2014-11
- Volume
- 82
- Issue
- 6
- Pages
- 2295-2326
- ISSN
- 0012-9682 (ISSN-L)
- OpenAlex
- W1537287213
- Cited by
- 3,036
- References
- 39
- Citations by year
- 2026: 293 · 2025: 344 · 2024: 327 · 2023: 321 · 2022: 317 · 2021: 295 · 2020: 285 · 2019: 220 · 2018: 194 · 2017: 197 · 2016: 117 · 2015: 80 · 2014: 31 · 2013: 11 · 2012: 1
- FWCI
- 149.58
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Econometrica (publishedVersion)
- Deep Blue (University of Michigan) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Keywords
- regression discontinuity design, confidence intervals, loess, bandwidth selection, fuzzy regression discontinuity, average treatment effect, bias correction, standard error estimation, empirical coverage, nonparametric inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
