The DOI matches its Crossref record; 56 of the 58 references it cites were found in a registry or an index.
Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends
Abstract
This paper discusses two important limitations of the common practice of testing for preexisting differences in trends (“ pre-trends”) when using difference-in-differences and related methods. First, conventional pre-trends tests may have low power. Second, conditioning the analysis on the result of a pretest can distort estimation and inference, potentially exacerbating the bias of point estimates and under-coverage of confidence intervals. I analyze these issues both in theory and in simulations calibrated to a survey of recent papers in leading economics journals, which suggest that these limitations are important in practice. I conclude with practical recommendations for mitigating these issues. (JEL A14, C23, C51)
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, conducting statistical pre-trend tests and conditioning the analysis on passing these tests may exacerbate bias in the estimated treatment effects because the observed draws of data that pass such tests are a select sample from the true underlying data-generating process [76].”
From Difference-in-Differences for Health Policy and Practice: A Review of Modern Methods (2025) · cites this paper as [76]
In the citing paper
“However, as several papers have noted [20, 24, 76-77], these tests can be misleading because they may not have adequate statistical power to detect a violation, even if there are non-parallel pre-intervention trend differences between treatment and comparison groups [20, 24, 76-77].”
From Difference-in-Differences for Health Policy and Practice: A Review of Modern Methods (2025) · cites this paper as [76]
In the citing paper
“As with tests of slope differences, event study estimates should not be evaluated based on whether confidence intervals overlap zero [24, 76].”
From Difference-in-Differences for Health Policy and Practice: A Review of Modern Methods (2025) · cites this paper as [76]
In the citing paper
“Moreover, our strategy circumvents the inference problems after pre-testing that were pointed out by Roth (2022), under spherical errors.”
From Revisiting Event-Study Designs: Robust and Efficient Estimation (2024) · cites this paper as Roth (2022)
In the citing paper
“This procedure is robust to treatment-effect heterogeneity and, under spherical errors, has attractive power properties and avoids the problem of inference after pre-testing explained by Roth (2022).”
From Revisiting Event-Study Designs: Robust and Efficient Estimation (2024) · cites this paper as Roth (2022)
16 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Difference-in-Differences with multiple time periods
Callaway, B., & Sant’Anna, P. H. C. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.12.001 · Cited by 9,126
Difference-in-differences with variation in treatment timing
Goodman-Bacon, A. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2021.03.014 · Cited by 8,046
Estimating dynamic treatment effects in event studies with heterogeneous treatment effects
Sun, L., & Abraham, S. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.09.006 · Cited by 5,518
Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme
Heckman, J. J., Ichimura, H., & Todd, P. E. (1997) · The Review of Economic Studies · doi:10.2307/2971733 · Cited by 5,486
Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects
de Chaisemartin, C., & D’Haultfœuille, X. (2020) · American Economic Review · doi:10.1257/aer.20181169 · Cited by 5,031
Semiparametric Difference-in-Differences Estimators
Abadie, A. (2005) · The Review of Economic Studies · doi:10.1111/0034-6527.00321 · Cited by 2,286
A More Credible Approach to Parallel Trends
Rambachan, A., & Roth, J. (2023) · Review of Economic Studies · doi:10.1093/restud/rdad018 · Cited by 1,559
Self-citationEstimating the Effect of Training Programs on Earnings
Ashenfelter, O. (1978) · The Review of Economics and Statistics · doi:10.2307/1924332 · Cited by 1,515
Doubly robust difference-in-differences estimators
Sant’Anna, P. H. C., & Zhao, J. (2020) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.06.003 · Cited by 1,047 · cited 1×
“…of this form (under suitable regularity conditions) include dynamic TWFE estimators, the GMM estimator of Freyaldenhoven, Hansen and Shapiro (2019), and methods for difference-in-differences conditional on covariates (Abadie, 2005; Heckman, Ichimura and Todd, 1997; Sant’Anna and Zhao, 2020).”
MethodDid Unilateral Divorce Laws Raise Divorce Rates? A Reconciliation and New Results
Wolfers, J. (2006) · American Economic Review · doi:10.1257/aer.96.5.1802 · Cited by 950
Research works printed but not linked: 3
- Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. New York: Routledge.
- Lee Jason D, 2016, Annals of Statistics, 44, 907
- Leeb Hannes, 2005, Econometric Theory, 21, 21
Cited by
Difference-in-Differences with multiple time periods
Callaway, B., & Sant’Anna, P. H. C. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.12.001 · Cited by 9,126
Estimating dynamic treatment effects in event studies with heterogeneous treatment effects
Sun, L., & Abraham, S. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.09.006 · Cited by 5,518
Revisiting Event-Study Designs: Robust and Efficient Estimation
Borusyak, K., Jaravel, X., & Spiess, J. (2024) · Review of Economic Studies · doi:10.1093/restud/rdae007 · Cited by 2,280
What's trending in difference-in-differences? A synthesis of the recent econometrics literature
Roth, J., Sant’Anna, P. H. C., Bilinski, A., & Poe, J. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2023.03.008 · Cited by 2,027
“A natural statistic to report in many contexts is the ‘‘breakdown’’ value of ¯ M using the sensitivity analysis in Rambachan and Roth (2022b) — i.e. how big would the post-treatment violation of parallel trends have to be relative to the largest pre-treatment violation to invalidate a particular conclusion?”
InfluentialSelf-citationA More Credible Approach to Parallel Trends
Rambachan, A., & Roth, J. (2023) · Review of Economic Studies · doi:10.1093/restud/rdad018 · Cited by 1,559
Self-citation
Metadata
- Authors
- Roth, J.
- Affiliations
- Brown University (US)
- Corresponding author
- Roth, Jonathan
- Venue
- American Economic Review: Insights
- Publisher
- American Economic Association
- Published
- 2022-09-01
- Volume
- 4
- Issue
- 3
- Pages
- 305-322
- ISSN
- 2640-205X (ISSN-L)
- 2640-2068
- OpenAlex
- W4293761886
- Semantic Scholar
- e997ce06ce3226f7377de5dc8d9cdd6989d501d0
- Cited by
- 769
- References
- 58
- Citations by year
- 2026: 151 · 2025: 190 · 2024: 177 · 2023: 109 · 2022: 70
- FWCI
- 100.67
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Economic Review Insights (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- difference-in-differences, parallel trends assumption, statistical power, confidence interval coverage, estimation bias, causal inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends. https://app.cochat.ai/papers/10.1257/aeri.20210236
- The paper
- Roth, J. (2022). Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends. American Economic Review: Insights. https://doi.org/10.1257/aeri.20210236
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.