The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
How Much Should We Trust Staggered Difference-In-Differences Estimates?
Open accessAbstract
Difference-in-differences analysis with staggered treatment timing is frequently used to assess the impact of policy changes on corporate outcomes in academic research. However, recent advances in econometric theory show that such designs are likely to be biased in the presence of treatment effect heterogeneity. Given the pronounced use of staggered treatment designs in applied corporate finance and accounting research, this finding potentially impacts a large swath of prior findings in these fields. We survey the nascent literature and document how and when such bias arises from treatment effect heterogeneity. We apply recently proposed methods to a set of prior published results, and find that correcting for the bias induced by the staggered nature of policy adoption frequently impacts the estimated effect from standard difference-in-difference studies. In many cases, the reported effects in prior research become indistinguishable from zero.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
How Much Should We Trust Differences-in-Differences Estimates?
Bertrand, M., Duflo, E., & Mullainathan, S. (2004) · The Quarterly Journal of Economics · doi:10.1162/003355304772839588 · Cited by 10,641
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Difference-in-Differences with multiple time periods
Callaway, B., & Sant’Anna, P. H. C. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.12.001 · Cited by 9,126
Difference-in-differences with variation in treatment timing
Goodman-Bacon, A. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2021.03.014 · Cited by 8,046
Estimating dynamic treatment effects in event studies with heterogeneous treatment effects
Sun, L., & Abraham, S. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.09.006 · Cited by 5,518
Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme
Heckman, J. J., Ichimura, H., & Todd, P. E. (1997) · The Review of Economic Studies · doi:10.2307/2971733 · Cited by 5,486
Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects
de Chaisemartin, C., & D’Haultfœuille, X. (2020) · American Economic Review · doi:10.1257/aer.20181169 · Cited by 5,031
Big bad banks? The winners and losers from bank deregulation in the United States
BECK, T., LEVINE, R., & LEVKOV, A. (2010) · The Journal of Finance · doi:10.1111/j.1540-6261.2010.01589.x · Cited by 3,358
The Effect of Minimum Wages on Low-Wage Jobs*
Cengiz, D., Dube, A., Lindner, A., & Zipperer, B. (2019) · The Quarterly Journal of Economics · doi:10.1093/qje/qjz014 · Cited by 2,688
Semiparametric Difference-in-Differences Estimators
Abadie, A. (2005) · The Review of Economic Studies · doi:10.1111/0034-6527.00321 · Cited by 2,286
Causal Inference Using Potential Outcomes: Design, Modeling, Decisions
Rubin, D. B. (2005) · Journal of the American Statistical Association · doi:10.1198/016214504000001880 · Cited by 2,173
Natural and Quasi- Experiments in Economics
Meyer, B. (1994) · Journal of Business and Economic Statistics · doi:10.2307/1392369 · Cited by 1,792
Doubly robust difference-in-differences estimators
Sant’Anna, P. H. C., & Zhao, J. (2020) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.06.003 · Cited by 1,047 · cited 45×
“…effect average, which is likely a result of using OLS varianceweighting rather than weighting explicitly by the sample share as in Callaway and Sant’Anna (2020) and Sun and Abraham (2020).18 The results of this stylized simulation show how in settings with panel data and heterogeneous…”
BackgroundMethodResultInfluentialDesign-based analysis in Difference-In-Differences settings with staggered adoption
Athey, S., & Imbens, G. W. (2022) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.10.012 · Cited by 917
Growing out of trouble? Corporate responses to liability risk
Gormley, T. A., & Matsa, D. A. (2011) · Review of Financial Studies · doi:10.1093/rfs/hhr011 · Cited by 690
On the Use of Two-Way Fixed Effects Regression Models for Causal Inference with Panel Data
Imai, K., & Kim, I. S. (2021) · Political Analysis · doi:10.1017/pan.2020.33 · Cited by 623 · cited 2×
“…where a treatment occurs across units at different points in time, 5The body of work examining these issues include: Athey and Imbens (2018), Borusyak and Jaravel (2017),Callaway and Sant’Anna (2020), Goodman-Bacon (2019), Imai and Kim (2020), Strezhnev (2018), and Sun and Abraham (2020).”
BackgroundRevisiting event study designs: robust and efficient estimation
(2022) · arXiv (preprint) · doi:10.47004/wp.cem.2022.1122 · Cited by 607
Who Is Screened Out? Application Costs and the Targeting of Disability Programs
Deshpande, M., & Li, Y. (2019) · American Economic Journal: Economic Policy · doi:10.1257/pol.20180076 · Cited by 534
Revisiting Event Study Designs
Borusyak, K., & Jaravel, X. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2826228 · Cited by 463
Pre-Event Trends in the Panel Event-Study Design
Freyaldenhoven, S., Hansen, C., & Shapiro, J. M. (2019) · American Economic Review · doi:10.1257/aer.20180609 · Cited by 427
Board reforms and firm value: worldwide evidence
Fauver, L., Hung, M., Li, X., & Taboada, A. G. (2017) · Journal of Financial Economics · doi:10.1016/j.jfineco.2017.04.010 · Cited by 407
Identification of and correction for publication bias
Andrews, I., & Kasy, M. (2019) · American Economic Review · doi:10.1257/aer.20180310 · Cited by 388
Institutional and Legal Context in Natural Experiments: The Case of State Antitakeover Laws
KARPOFF, J. M., & WITTRY, M. D. (2018) · The Journal of Finance · doi:10.1111/jofi.12600 · Cited by 272
Staggeringly Problematic: A Primer on Staggered DiD for Accounting Researchers
Barrios, J. M. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3794859 · Cited by 227
Manski, C. F., & Pepper, J. V. (2018) · Review of Economics and Statistics · doi:10.1162/rest_a_00689 · Cited by 158
Simultaneous confidence bands: Theory, implementation, and an application to SVARs
Montiel Olea, J. L., & Plagborg‐Møller, M. (2019) · Journal of Applied Econometrics · doi:10.1002/jae.2656 · Cited by 153
When Is Parallel Trends Sensitive to Functional Form?
Roth, J., & Sant'Anna, P. H. C. (2023) · Econometrica · doi:10.3982/ecta19402 · Cited by 142
Simple Diagnostics for Two-Way Fixed Effects
Jakiela, P. (2021) · doi:10.36934/wecon:2021-05 · Cited by 95
Significance testing in empirical finance: a critical review and assessment
Kim, J. H., & Ji, P. I. (2015) · Journal of Empirical Finance · doi:10.1016/j.jempfin.2015.08.006 · Cited by 86
Real effects of share repurchases legalization on corporate behaviors
Wang, Z., Yin, Q. E., & Yu, L. (2020) · Journal of Financial Economics · doi:10.1016/j.jfineco.2020.10.008 · Cited by 75
Short-Termism and Capital Flows
Fried, J. M., & Wang, C. C. Y. (2017) · The Review of Corporate Finance Studies · doi:10.1093/rcfs/cfy011 · Cited by 71
Self-citationWho Is Screened Out? Application Costs and the Targeting of Disability Programs
Deshpande, M., & Li, Y. (2017) · National Bureau of Economic Research · doi:10.3386/w23472 · Cited by 26
Doubly Robust Difference-in-Differences EstimatorsPreprint
Sant’Anna, P. H. C., & Zhao, J. B. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3293315 · Cited by 24
Short-Termism, Shareholder Payouts, and Investment in the EU
Fried, J. M., & Wang, C. C. Y. (2020) · European Financial Management · doi:10.1111/eufm.12305 · Cited by 19
Self-citationZdrojewski, A., & Butler, A. W. (2021) · Critical Finance Review · doi:10.1561/104.00000147 · Cited by 1
Cited by
Revisiting Event-Study Designs: Robust and Efficient Estimation
Borusyak, K., Jaravel, X., & Spiess, J. (2024) · Review of Economic Studies · doi:10.1093/restud/rdae007 · Cited by 2,280
What's trending in difference-in-differences? A synthesis of the recent econometrics literature
Roth, J., Sant’Anna, P. H. C., Bilinski, A., & Poe, J. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2023.03.008 · Cited by 2,027
Revisiting event study designs: robust and efficient estimation
(2022) · arXiv (preprint) · doi:10.47004/wp.cem.2022.1122 · Cited by 607
“See also Baker et al. (2021) for evidence that using robust, rather than regression-based, event study estimation matters in other empirical contexts.”
BackgroundSchmidheiny, K., & Siegloch, S. (2023) · Journal of Applied Econometrics · doi:10.1002/jae.2971 · Cited by 293
Simple Diagnostics for Two-Way Fixed Effects
Jakiela, P. (2021) · doi:10.36934/wecon:2021-05 · Cited by 95
Metadata
- Authors
- Baker, A., Larcker, D. F., & Wang, C. C. Y.
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2021
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W3134701292
- Cited by
- 670
- References
- 35
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.