CoChat
Sign Up
PreprintCited by 243Advanced Causal Inference TechniquesSpatial and Panel Data AnalysisStatistical Methods and Bayesian Inference

Difference-in-Differences with a Continuous Treatment

Open access
Callaway, Brantly, Goodman-Bacon, Andrew, Sant'Anna, Pedro H. C.·2021·arXiv
DOI 10.48550/arxiv.2107.02637arXiv 2107.02637
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper analyzes difference-in-differences designs with a continuous treatment. We show that treatment-on-the-treated-type parameters are identified under a parallel trends assumption analogous to the binary treatment case. However, comparing these parameters across treatments is challenging because parallel trends does not rule out selection bias. We discuss alternative, typically stronger, assumptions that eliminate selection bias. We further show that popular two-way fixed effects estimands admit multiple interpretations, depending on the underlying causal building block, all having important limitations as meaningful summaries of treatment effects. Finally, we introduce alternative estimation procedures that avoid these drawbacks and demonstrate them in an empirical application.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2107.02637

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • de Chaisemartin and D'Haultfœuille (2021) and Callaway, Goodman-Bacon and Sant'Anna (2021) study settings with multi-valued or continuous treatments.

    “Finally, de Chaisemartin and D'Haultfœuille (2021) and Callaway, Goodman-Bacon and Sant'Anna (2021) study settings with multi-valued or continuous treatments.”

    From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Callaway, Goodman-Bacon and Sant'Anna (2021)

  • The TWFE estimator is sensitive to effect heterogeneity, notably when using continuous treatments.

    “the estimator is sensitive to effect heterogeneity, notably when using continuous treatments (Callaway, Goodman-Bacon and Sant'Anna, 2021).”

    From Extension and management pathways for enhanced farm sustainability: evidence from Irish dairy farms (2023) · cites this paper as Callaway, Goodman-Bacon and Sant'Anna, 2021

  • TWFE estimates can give a misleading summary of the overall average causal response because the estimator puts more weight on doses close to the average than on the tails.

    “TWFE estimates can give a misleading summary of the overall average causal response because the estimator puts more weight on doses close to the average than on the tails.”

    From Extension and management pathways for enhanced farm sustainability: evidence from Irish dairy farms (2023) · cites this paper as Callaway, Goodman-Bacon and Sant'Anna (2021)

  • When estimating the intensity of a continuous treatment with the TWFE estimator, the obtained effect mixes both the level effect and the slope effect.

    “when estimating the intensity of a continuous treatment d with the TWFE estimator (e.g.𝐷𝐺 in equations (1 and 2)), the obtained effect mixes both the 'level effect' and the 'slope effect'.”

    From Extension and management pathways for enhanced farm sustainability: evidence from Irish dairy farms (2023) · cites this paper as Callaway, Goodman-Bacon and Sant'Anna (2021)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Research works printed but not linked: 45

  1. Acemoglu, Daron and Amy Finkelstein (2008). "Input and technology choices in regulated industries: Evidence from the health care sector". Journal of Political Economy 116.5, pp. 837-880.
  2. Ackerberg, Daniel, Xiaohong Chen, and Jinyong Hahn (2012). "A practical asymptotic variance estimator for two-step semiparametric estimators". Review of Economics and Statistics 94.2, pp. 481-498.
  3. Ai, Chunrong and Xiaohong Chen (2007). "Estimation of possibly misspecified semiparametric conditional moment restriction models with different conditioning variables". Journal of Econometrics 141, pp. 5-43.
  4. Angrist, Joshua D (1998). "Estimating the labor market impact of voluntary military service using Social Security data on military applicants". Econometrica 66.2, pp. 249-288.
  5. Angrist, Joshua D and Ivan Fernandez-Val (2013). "ExtrapoLATE-ing: External validity and overidentification in the LATE framework". Advances in Economics and Econometrics: Volume 3, Econometrics: Tenth World Congress. Vol. 51. Cambridge University Press, pp. 401-434.
  6. Angrist, Joshua D, Kathryn Graddy, and Guido W Imbens (2000). "The interpretation of instrumental variables estimators in simultaneous equations models with an application to the demand for fish". The Review of Economic Studies 67.3, pp. 499-527.
  7. Angrist, Joshua D and Guido W Imbens (1995). "Two-stage least squares estimation of average causal effects in models with variable treatment intensity". Journal of the American Statistical Association 90.430, pp. 431- 442.
  8. Angrist, Joshua D and Jorn-Steffen Pischke (2008). Mostly Harmless Econometrics: An Empiricist's Compan- ion. Princeton University Press.
  9. Aronow, Peter M and Cyrus Samii (2016). "Does regression produce representative estimates of causal effects?" American Journal of Political Science 60.1, pp. 250-267.
  10. Athey, Susan and Guido Imbens (2006). "Identification and inference in nonlinear difference-in-differences models". Econometrica 74.2, pp. 431-497.
  11. Borusyak, Kirill, Xavier Jaravel, and Jann Spiess (2023). "Revisiting event study designs: Robust and efficient estimation". Review of Economic Studies Forthcoming.
  12. Callaway, Brantly (2023). "Difference-in-differences for policy evaluation". Handbook of Labor, Human Re- sources and Population Economics. Ed. by Klaus F. Zimmermann. Springer International Publishing, pp. 1- 61.
  13. Callaway, Brantly and Pedro H. C. Sant'Anna (2021). "Difference-in-differences with multiple time periods". Journal of Econometrics 225.2, pp. 200-230.
  14. Cameron, A. Colin and Pravin K. Trivedi (2005). Microeconometrics: Methods and Applications. Cambridge University Press.
  15. Cattaneo, Matias, Luke Keele, Rocío Titiunik, and Gonzalo Vazquez-Bare (2021). "Extrapolating treatment effects in multi-cutoff regression discontinuity designs". Journal of the American Statistical Association 116.536, pp. 1941-1952.
  16. Cattaneo, Matias, Rocío Titiunik, Gonzalo Vazquez-Bare, and Luke Keele (2016). "Interpreting regression discontinuity designs with multiple cutoffs". Journal of Politics 78.4, pp. 1229-1248.
  17. Chen, Jiafeng, Xiaohong Chen, and Elie Tamer (2023). "Efficient Estimation in NPIV Models: A Comparison of Various Neural Networks-Based Estimators". Journal of Econometrics 235.2, pp. 1848-187.
  18. Chen, Xiaohong, Timothy Christensen, and Sid Kankanala (2023). "Adaptive estimation and uniform confi- dence bands for nonparametric structural functions and elasticities". Review of Economic Studies Forth- coming.
  19. Chen, Xiaohong and Timothy M. Christensen (2015). "Optimal uniform convergence rates and asymptotic normality for series estimators under weak dependence and weak conditions". Journal of Econometrics 188.2, pp. 447-465.
  20. -(2018). "Optimal sup-norm rates and uniform inference on nonlinear functionals of nonparametric IV regression". Quantitative Economics 9.1, pp. 39-84.
  21. Chodorow-Reich, Gabriel, Plamen T. Nenov, and Alp Simsek (2021). "Stock market wealth and the real economy: A local labor market approach". American Economic Review 111.5, pp. 1613-57.
  22. D'Haultfœuille, Xavier, Stefan Hoderlein, and Yuya Sasaki (2023). "Nonparametric difference-in-differences in repeated cross-sections with continuous treatments". Journal of Econometrics 234.2, pp. 664-690.
  23. de Chaisemartin, Clement and Xavier D'Haultfœuille (2018). "Fuzzy differences-in-differences". The Review of Economic Studies 85.2, pp. 999-1028.
  24. -(2020). "Two-way fixed effects estimators with heterogeneous treatment effects". American Economic Re- view 110.9, pp. 2964-2996.
  25. de Chaisemartin, Clément and Xavier d'Haultfoeuille (2023). "Two-way fixed effects and differences-in- differences with heterogeneous treatment effects: A survey". The Econometrics Journal 26.3, pp. C1- C30.
  26. Fricke, Hans (2017). "Identification based on difference-in-differences approaches with multiple treatments". Oxford Bulletin of Economics and Statistics 79.3, pp. 426-433.
  27. Goldsmith-Pinkham, Paul, Isaac Sorkin, and Henry Swift (2020). "Bartik instruments: What, when, why, and how". The American Economic Review 110.8, pp. 2586-2624.
  28. Goodman-Bacon, Andrew (2021). "Difference-in-differences with variation in treatment timing". Journal of Econometrics 225.2, pp. 254-277.
  29. Hendren, Nathaniel (2016). "The policy elasticity". Tax Policy and the Economy 30.1, pp. 51-89.
  30. Hill, Sir Austin Bradford (1965). "The environment and disease: association or causation?" Journal of the Royal Society of Medicine 58.5, pp. 295-300.
  31. Low, Mark G. (1997). "On nonparametric confidence intervals". Annals of Statistics 25.6, pp. 2547-2554.
  32. Marcus, Michelle and Pedro H. C. Sant'Anna (2021). "The role of parallel trends in event study settings: An application to environmental economics". Journal of the Association of Environmental and Resource Economists 8.2, pp. 235-275.
  33. Meyer, Bruce D. (1995). "Natural and quasi-experiments in economics". Journal of Business & Economic Statistics 13.2, pp. 151-161.
  34. Mogstad, Magne, Andres Santos, and Alexander Torgovitsky (2018). "Using instrumental variables for inference about policy relevant treatment parameters". Econometrica 86.5, pp. 1589-1619.
  35. Newey, Whitney K. (1994). "The asymptotic variance of semiparametric estimators". Econometrica, pp. 1349- 1382.
  36. Newey, Whitney K. and Thomas M. Stoker (1993). "Efficiency of weighted average derivative estimators and index models". Econometrica 61.5, pp. 1199-1223.
  37. Oreopoulos, Philip (2006). "Estimating average and local average treatment effects of education when compul- sory schooling laws really matter". American Economic Review 96.1, pp. 152-175.
  38. Roth, Jonathan (2022). "Pretest with caution: Event-study estimates after testing for parallel trends". Amer- ican Economic Review: Insights 4.3, pp. 305-322.
  39. Roth, Jonathan, Pedro H. C. Sant'Anna, Alyssa Bilinski, and John Poe (2023). "What's trending in difference- in-differences? A synthesis of the recent econometrics literature". Journal of Econometrics 235.2, pp. 2218- 2244.
  40. Saez, Emmanuel, Joel Slemrod, and Seth H. Giertz (2012). "The elasticity of taxable income with respect to marginal tax rates: A critical review". Journal of Economic Literature 50.1, pp. 3-50.
  41. S loczyński, Tymon (2022a). "Interpreting OLS estimands when treatment effects are heterogeneous: Smaller groups get larger weights". The Review of Economics and Statistics 104.3, pp. 501-509.
  42. Sun, Liyang and Sarah Abraham (2021). "Estimating dynamic treatment effects in event studies with hetero- geneous treatment effects". Journal of Econometrics 225.2, pp. 175-199.
  43. Sun, Liyang and Jesse M. Shapiro (2022). "A linear panel model with heterogeneous coefficients and variation in exposure". Journal of Economic Perspectives 36.4, pp. 193-204.
  44. Wooldridge, Jeffrey M (2010). Econometric Analysis of Cross Section and Panel Data. MIT press.
  45. Yitzhaki, Shlomo (1996). "On using linear regressions in welfare economics". Journal of Business & Economic Statistics 14.4, pp. 478-486.

Cited by

Sorted by citations

Related

Authors
Callaway, B., Goodman-Bacon, A., & Sant'Anna, P. H. C.
Affiliations
  • University of Georgia (US)
  • National Bureau of Economic Research (US)
ORCID
Callaway, Brantly
Venue
arXiv
Publisher
arXiv
Published
2021
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2107.02637
arXiv
2107.02637
OpenAlex
W3186938411
Cited by
243
References
0
Citations by year
2026: 5 · 2025: 23 · 2024: 65 · 2023: 94 · 2022: 42 · 2021: 10 · 2020: 2 · 2019: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
difference-in-differences, continuous treatment, parallel trends assumption, selection bias, two-way fixed effects, causal identification, treatment effect estimation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review