CoChat
Sign Up
Cited by 757Advanced Causal Inference TechniquesSpatial and Panel Data AnalysisStatistical Methods and Bayesian Inference

Difference-in-Differences with Variation in Treatment Timing

Open access
Goodman-Bacon, Andrew·2018·National Bureau of Economic Research
DOI 10.3386/w25018
Open this paper in CoChat — free account

Abstract

The canonical difference-in-differences (DD) model contains two time periods, "pre" and "post", and two groups, "treatment" and "control".Most DD applications, however, exploit variation across groups of units that receive treatment at different times.This paper derives an expression for this general DD estimator, and shows that it is a weighted average of all possible two-group/ two-period DD estimators in the data.This result provides detailed guidance about how to use regression DD in practice.I define the DD estimand and show how it averages treatment effect heterogeneity and that it is biased when effects change over time.I propose a new balance test derived from a unified definition of common trends.I show how to decompose the difference between two specifications, and I apply it to models that drop untreated units, weight, disaggregate time fixed effects, control for unit-specific time trends, or exploit a third difference.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3386/w25018

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Except for a few specific scenarios, the static TWFE model does not produce a consistent estimate for the overall average treatment effect.

    “The natural question is, 'does the static TWFE model, (1), produce a consistent estimate for the overall average treatment effect?' Except for a few specific scenarios, the answer is no (Sun and Abraham 2020; Goodman-Bacon 2018; Borusyak, Jaravel, and Spiess 2021; Chaisemartin and D'Haultfoeuille 2019).”

    From did2s: Two-Stage Difference-in-Differences (2022) · cites this paper as Goodman-Bacon 2018

  • A rapidly growing econometric literature has identified difficulties in traditional difference-in-differences estimation when treatment turns on at different times for different groups and when treatment effects vary across groups and over time.

    “Arapidly growing econometric literature has identified difficulties in traditional difference-in-differences estimation when treatment turns on at different times for different groups and when the effects of treatment vary across groups and over time (Callaway and Sant'Anna 2020; Sun and Abraham 2020; Goodman-Bacon 2018; Borusyak, Jaravel, and Spiess 2021; Chaisemartin and D'Haultfoeuille 2019).”

    From did2s: Two-Stage Difference-in-Differences (2022) · cites this paper as Goodman-Bacon 2018

  • When treatment adoption is staggered and average treatment effects vary across groups and over time, difference-in-differences regression does not identify an easily interpretable measure of the typical effect of the treatment.

    “A recent literature has shown that when adoption of a treatment is staggered and average treatment effects vary across groups and over time, difference-in-differences regression does not identify an easily interpretable measure of the typical effect of the treatment.”

    From Two-stage differences in differences (2022) · cites this paper as Goodman-Bacon, 2018

  • Goodman-Bacon (2018) shows that the regression difference-in-differences estimate represents a weighted average of all two-group, two-period differences in differences, which under parallel trends identifies a combination of weighted averages of group × period-specific average treatment effects and changes over time in those effects.

    “Goodman-Bacon (2018) further shows that the regression difference-in-differences estimate represents a weighted average of all two-group, two-period differences in differences, which under parallel trends identifies a combination of weighted averages of group × period-specific average treatment effects and changes over time in those effects.”

    From Two-stage differences in differences (2022) · cites this paper as Goodman-Bacon (2018)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 68 · Cited by 757

Research works printed but not linked: 45

  1. Abadie, Alberto, Alexis Diamond, and Jens Hainmueller. 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program." Journal of the American Statistical Association 105 (490):493-505. doi: 10.1198/jasa.2009.ap08746. · DOI · Paper page
  2. Allcott, Hunt. 2015. "Site Selection Bias in Program Evaluation." The Quarterly Journal of Economics 130 (3):1117-1165. doi: 10.1093/qje/qjv015. · DOI · Paper page
  3. Almond, Douglas, Hilary W. Hoynes, and Diane Whitmore Schanzenbach. 2011. "Inside the War On Poverty: The Impact of Food Stamps on Birth Outcomes." The Review of Economics and Statistics 93 (2):387-403. doi: 10.2307/23015943. · DOI
  4. Angrist, Joshua D. 1991. "Grouped-data estimation and testing in simple labor-supply models." Journal of Econometrics 47 (2):243-266. doi: https://doi.org/10.1016/0304-4076(91)90101-I. · DOI
  5. Angrist, Joshua D., and Alan B. Krueger. 1999. "Chapter 23 -Empirical Strategies in Labor Economics." In Handbook of Labor Economics, edited by Orley C. Ashenfelter and David Card, 1277-1366. Elsevier.
  6. Angrist, Joshua David, and Jörn-Steffen Pischke. 2009. Mostly harmless econometrics : an empiricist's companion. Princeton: Princeton University Press.
  7. Angrist, Joshua David, and Jörn-Steffen Pischke. 2015. Mastering 'metrics : the path from cause to effect. Princeton ; Oxford: Princeton University Press.
  8. Athey, Susan, and Guido W. Imbens. 2006. "Identification and Inference in Nonlinear Difference-in-Differences Models." Econometrica 74 (2):431-497.
  9. Bailey, Martha J., and Andrew Goodman-Bacon. 2015. "The War on Poverty's Experiment in Public Medicine: Community Health Centers and the Mortality of Older Americans." American Economic Review 105 (3):1067-1104.
  10. Besley, Timothy, and Anne Case. 2002. "Unnatural Experiments? Estimating the Incidence of Endogenous Policies." The Economic Journal 110 (467):672-694. doi: 10.1111/1468-0297.00578. · DOI
  11. Bhuller, Manudeep, Tarjei Havnes, Edwin Leuven, and Magne Mogstad. 2013. "Broadband Internet: An Information Superhighway to Sex Crime?" The Review of Economic Studies 80 (4 (285)):1237-1266.
  12. Bitler, Marianne P., Jonah B. Gelbach, and Hilary W. Hoynes. 2003. "Some Evidence on Race, Welfare Reform, and Household Income." The American Economic Review 93 (2):293-298. doi: 10.2307/3132242. · DOI
  13. Blinder, Alan S. 1973. "Wage Discrimination: Reduced Form and Structural Estimates." The Journal of Human Resources 8 (4):436-455. doi: 10.2307/144855. · DOI · Paper page
  14. Callaway, Brantly, Tong Li, and Tatsushi Oka. forthcoming. "Quantile Treatment Effects in Difference in Differences Models under Dependence Restrictions and with only Two Time Periods." Journal of Econometrics.
  15. Cameron, Adrian Colin, and P. K. Trivedi. 2005. Microeconometrics : methods and applications. Cambridge ; New York: Cambridge University Press.
  16. Chernozhukov, Victor, Iván Fernández-Val, Jinyong Hahn, and Whitney Newey. 2013. "Average and Quantile Effects in Nonseparable Panel Models." Econometrica 81 (2):535-580. doi: 10.3982/ECTA8405. · DOI · Paper page
  17. Chyn, Eric. forthcoming. "Moved to Opportunity: The Long-Run Effect of Public Housing Demolition on Labor Market Outcomes of Children." American Economic Review.
  18. de Chaisemartin, C., and X. D'HaultfŒuille. 2018a. "Fuzzy Differences-in-Differences." The Review of Economic Studies 85 (2):999-1028. doi: 10.1093/restud/rdx049. · DOI · Paper page
  19. Deaton, Angus. 1997. The Analysis of Household Surveys : a Microeconometric Approach to Development Policy. Baltimore, MD: Johns Hopkins University Press.
  20. Deshpande, Manasi, and Yue Li. 2017. "Who Is Screened Out? Application Costs and the Targeting of Disability Programs." National Bureau of Economic Research Working Paper Series No. 23472. doi: 10.3386/w23472. · DOI
  21. Fadlon, Itzik, and Torben Heien Nielsen. 2015. "Family Labor Supply Responses to Severe Health Shocks." National Bureau of Economic Research Working Paper Series No. 21352. doi: 10.3386/w21352. · DOI
  22. Freyaldenhoven, Simon, Christian Hansen, and Jesse M. Shapiro. 2018. "Pre-event Trends in the Panel Event-study Design." National Bureau of Economic Research Working Paper Series No. 24565. doi: 10.3386/w24565. · DOI
  23. Frisch, Ragnar, and Frederick V. Waugh. 1933. "Partial Time Regressions as Compared with Individual Trends." Econometrica 1 (4):387-401. doi: 10.2307/1907330. · DOI · Paper page
  24. Gibbons, Charles, E., Juan Carlos Suárez Serrato, and Michael Urbancic, B. 2018. Broken or Fixed Effects? In Journal of Econometric Methods.
  25. Goodman, Joshua. 2017. "The Labor of Division: Returns to Compulsory High School Math Coursework." National Bureau of Economic Research Working Paper Series No. 23063. doi: 10.3386/w23063. · DOI
  26. Heckman, James J., Robert J. Lalonde, and Jeffrey A. Smith. 1999. "Chapter 31 -The Economics and Econometrics of Active Labor Market Programs." In Handbook of Labor Economics, edited by Orley C. Ashenfelter and David Card, 1865-2097. Elsevier.
  27. Holland, Paul W. 1986. "Statistics and Causal Inference." Journal of the American Statistical Association 81 (396):945-960. doi: 10.2307/2289064. · DOI · Paper page
  28. Jacobson, Louis S., Robert J. LaLonde, and Daniel G. Sullivan. 1993. "Earnings Losses of Displaced Workers." The American Economic Review 83 (4):685-709. doi: 10.2307/2117574. · DOI
  29. Joseph Hotz, V., Guido W. Imbens, and Julie H. Mortimer. 2005. "Predicting the efficacy of future training programs using past experiences at other locations." Journal of Econometrics 125 (1):241-270. doi: https://doi.org/10.1016/j.jeconom.2004.04.009. · DOI · Paper page
  30. Kitagawa, Evelyn M. 1955. "Components of a Difference Between Two Rates." Journal of the American Statistical Association 50 (272):1168-1194. doi: 10.2307/2281213. · DOI
  31. Lee, David S., and Thomas Lemieux. 2010. "Regression Discontinuity Designs in Economics." Journal of Economic Literature 48 (2):281-355.
  32. Malkova, Olga. 2017. "Can Maternity Benefits Have Long-Term Effects on Childbearing? Evidence From Soviet Russia." The Review of Economics and Statistics. doi: 10.1162/REST_a_00713. · DOI
  33. Meer, Jonathan, and Jeremy West. 2013. "Effects of the Minimum Wage on Employment Dynamics." National Bureau of Economic Research Working Paper Series No. 19262. doi: 10.3386/w19262. · DOI · Paper page
  34. Meyer, Bruce D. 1995. "Natural and Quasi-Experiments in Economics." Journal of Business & Economic Statistics 13 (2):151-161. doi: 10.2307/1392369. · DOI
  35. Neumark, David, J. M. Ian Salas, and William Wascher. 2014. "Revisiting the Minimum Wage-Employment Debate: Throwing Out the Baby with the Bathwater?" ILR Review 67 (3_suppl):608-648. doi: 10.1177/00197939140670S307. · DOI
  36. Oaxaca, Ronald. 1973. "Male-Female Wage Differentials in Urban Labor Markets." International Economic Review 14 (3):693-709. doi: 10.2307/2525981. · DOI · Paper page
  37. Oster, Emily. 2016. "Unobservable Selection and Coefficient Stability: Theory and Evidence." Journal of Business & Economic Statistics:1-18. doi: 10.1080/07350015.2016.1227711. · DOI · Paper page
  38. Pei, Zhuan, Jörn-Steffen Pischke, and Hannes Schwandt. 2017. "Poorly Measured Confounders are More Useful on the Left Than on the Right." National Bureau of Economic Research Working Paper Series No. 23232. doi: 10.3386/w23232. · DOI
  39. Shore-Sheppard, Lara D. . 2009. "Stemming the Tide? The Effect of Expanding Medicaid Eligibility On Health Insurance Coverage." The B.E. Journal of Economic Analysis & Policy 8 (2).
  40. Solon, Gary, Steven J. Haider, and Jeffrey M. Wooldridge. 2015. "What Are We Weighting For?" Journal of Human Resources 50 (2):301-316.
  41. Stevenson, Betsey, and Justin Wolfers. 2006. "Bargaining in the Shadow of the Law: Divorce Laws and Family Distress." The Quarterly Journal of Economics 121 (1):267-288.
  42. Walters, Christopher R. forthcoming. "The Demand for Effective Charter Schools." Journal of Political Economy.
  43. Wolfers, Justin. 2006. "Did Unilateral Divorce Laws Raise Divorce Rates? A Reconciliation and New Results." American Economic Review 96 (5):1802-1820.
  44. Wooldridge, Jeffrey M. 2005. "Fixed-Effects and Related Estimators for Correlated Random-Coefficient and Treatment-Effect Panel Data Models." The Review of Economics and Statistics 87 (2):385-390.
  45. Wooldridge, Jeffrey M. 2010. Econometric analysis of cross section and panel data. 2nd ed. Cambridge, Mass.: MIT Press.

Authors
Goodman-Bacon, A.
Affiliations
National Bureau of Economic Research (US)
Corresponding author
Goodman-Bacon, Andrew
Venue
National Bureau of Economic Research
Publisher
National Bureau of Economic Research
Published
2018-09
DOI
10.3386/w25018
OpenAlex
W3122538646
Cited by
757
References
68
Citations by year
2026: 13 · 2025: 32 · 2024: 55 · 2023: 57 · 2022: 105 · 2021: 232 · 2020: 180 · 2019: 51 · 2018: 27 · 2017: 4 · 2016: 1
Type
report
Language
en
Open access
Locations
National Bureau of Economic Research (publishedVersion)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
difference-in-differences, treatment timing, staggered treatment adoption, treatment effect heterogeneity, balance testing, time-varying treatment effects, time fixed effects

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review