CoChat
Sign Up
Cited by 192Advanced Causal Inference TechniquesSpatial and Panel Data AnalysisClimate Change Policy and Economics

The Role of Parallel Trends in Event Study Settings: An Application to Environmental Economics

Open access
Marcus, Michelle, Sant’Anna, Pedro H. C.·2021·Journal of the Association of Environmental and Resource Economists
DOI 10.1086/711509
Open this paper in CoChat — free accountOpen PDF

Abstract

Difference-in-differences (DID) research designs usually rely on variation of treatment timing such that, after making an appropriate parallel trends assumption, one can identify, estimate, and make inference about causal effects. In practice, however, different DID procedures rely on different parallel trends assumptions (PTAs), and recover different causal parameters. In this paper, we focus on staggered DID (also referred as event studies) and discuss the role played by the PTA in terms of identification and estimation of causal parameters. We document a “robustness” versus “efficiency” trade-off in terms of the strength of the underlying PTA and argue that practitioners should be explicit about these trade-offs whenever using DID procedures. We propose new DID estimators that reflect these trade-offs and derive their large sample properties. We illustrate the practical relevance of these results by assessing whether the transition from federal to state management of the Clean Water Act affects compliance rates.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper clarifies how different parallel trends assumptions (PTAs) in staggered difference-in-differences/event-study designs imply different identifying restrictions, parameters, and estimators, and documents a robustness-versus-efficiency trade-off. It proposes GMM-efficient DID estimators and a simpler “not-yet-treated” estimator, then applies them to the Clean Water Act transition, showing that conclusions about corruption and violation rates depend on the PTA invoked.

Key findings

  • The PTA used by Sun and Abraham (2020) and de Chaisemartin and D’Haultfœuille (2020) imposes parallel pre-treatment trends and allows any not-yet-treated group as a valid comparison; Callaway and Sant’Anna’s (2020) “never-treated” PTA does not restrict pre-trends and is weaker, while their “not-yet-treated” PTA is weaker than the former but still restricts some pre-trends. All can recover the same variety of average treatment effects.
  • Stronger PTAs permit more efficient estimators: under PTA 2.7, the proposed GMM estimator is semiparametrically efficient (Proposition 4.1), and the overidentifying restrictions can be tested with a Sargan-Hansen J-test. The proposed simpler estimator \(ATT^{ny+}(g,t)\) uses all not-yet-treated groups, does not explicitly restrict pre-trends, and is consistent and asymptotically normal (Proposition 5.2).
  • In the Clean Water Act application, baseline effects of state authorization on violation rates are close to zero across PTAs: Table 6.1 reports \(ATT^{simple}\) of -0.017 (never-treated), -0.010 (not-yet-treated), -0.003 (all not-yet-treated), and TWFE -0.003; \(\delta^{e,avg}\) estimates are -0.015, -0.008, and -0.003.
  • Corruption results depend on the PTA: allowing corruption-specific trends yields essentially no evidence that treatment effects vary by corruption (Table 6.2 \(ATT^{simple,1-0}\): -0.007, -0.008, -0.014), while not allowing corruption-specific trends yields larger negative estimates (-0.035, -0.035, -0.033), in line with the TWFE result.

Methods

  • Theoretical econometric comparison of staggered DID/event-study PTAs and parameters, focusing on Sun and Abraham, Callaway and Sant’Anna, and de Chaisemartin and D’Haultfœuille, using a stylized four-period example with groups first treated at time 3, time 4, and never-treated.
  • Proposes GMM estimators that exploit all moment restrictions implied by a chosen PTA, and derives large-sample properties; also proposes a simple plug-in DID estimator \(ATT^{ny+}(g,t)\) based on a weaker not-yet-treated PTA.
  • Empirical application replicates Grooms (2015) on the Clean Water Act: state-year panel, 23 treated states plus four never-treated states, 27 always-treated states dropped, 16 treatment groups and 33 time periods, 759 state-year pairs; violation rates weighted by total facilities, standard errors clustered at state level, event-study estimates with 20 leads/lags reported, and 1,000 bootstrap draws for simultaneous confidence intervals.

Limitations

  • Stated: GMM implementation can be challenging when the number of groups/time periods is large; in the application the GMM involves 780 moments with 195 overidentification restrictions while sample size is 759, so the authors do not use it.
  • Stated: PTA 2.6 is just-identified and cannot be directly tested; PTA 5.1 cannot be directly tested, so pre-treatment estimates provide only indirect/placebo-type evidence; PTA 2.7 is not directly testable via all pre-treatment event-study estimates, though specific restrictions can be tested. The authors also warn against specification search based on pre-tests due to multiple-testing distortions.
  • Evident from the text: the framework assumes panel data, binary and irreversible staggered treatment, no anticipation, and no covariates; the empirical corruption analysis notes that corruption is not randomly assigned, and the authors argue that allowing corruption-specific trends is the most natural identification setup in that context.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1086/711509

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 51 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Under a parallel trends assumption that allows corruption-specific trends, there is essentially no evidence that treatment effects vary by state corruption level.

    “For instance, when one assumes that, in the absence of treatment, the counterfactual outcome trends differ depending on whether a state has a long prevalence of corruption or not (“corruption-specific trends”), we find essentially no evidence that the treatment effects vary depending on whether a state is more or less corrupt.”
    Quote in the paperSubstantial claim
  • Findingown data

    The conclusion that violation rates dropped more in more corrupt states than in less corrupt states depends on the type of parallel trends assumption imposed.

    “By proceeding in this manner, we find that the conclusion that violation rates dropped more in more corrupt states than less corrupt states depends on the type of PTA imposed.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Grooms (2015)

      Not checked in the free run.

    • The authors state this as an assumption: In the absence of treatment, the expected outcome path is the same across all groups and all time periods.
    • The authors state this as an assumption: In the absence of treatment, the expected outcome change for a group first treated at g equals that for never-treated units.
    • The authors state this as an assumption: In the absence of treatment, the expected outcome change for a group first treated at g equals that for units not yet treated by time t.
  • Findingown data

    Table 6.2 indicates that under PTAs 6.4-6.6, there is evidence of a large post-authorization decline in violation rates for corrupt states relative to non-corrupt states.

    “when one relies on the”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: For each treatment group and corruption status, untreated outcome changes equal those for never-treated units, without conditioning on corruption status.
    • The authors state this as an assumption: For each treatment group and corruption status, untreated outcome changes equal those for not-yet-treated units, without conditioning on corruption status.
    • The authors state this as an assumption: For each treatment group and corruption status, untreated outcome changes equal those for units not yet treated at time t, without conditioning on corruption status.
  • Findingown data

    The aggregated summary measures point to an essentially null effect of state authorization on violation rates.

    “Essentially, all these summary measures indicates that state authorization has close to zero effect on violation”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Grooms (2015)

      Not checked in the free run.

  • Findingown data

    Under PTAs 6.4-6.6, which rule out corruption-specific trends, the event-study estimates indicate a sizable post-authorization drop in violation rates for corrupt states compared with non-corrupt states.

    “when one invokes Assumption 6.4 (Panel (b)), Assumption 6.5 (Panel (c)), or Assumption 6.6 (Panel (d)) and therefore rules out differential potential trends between corrupt and non-corrupt states, one finds that corrupt states experience a large decrease in violation rates after state authorization relative to non-corrupt states.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: For each treatment group and corruption status, untreated outcome changes equal those for never-treated units, without conditioning on corruption status.
    • The authors state this as an assumption: For each treatment group and corruption status, untreated outcome changes equal those for not-yet-treated units, without conditioning on corruption status.
    • The authors state this as an assumption: For each treatment group and corruption status, untreated outcome changes equal those for units not yet treated at time t, without conditioning on corruption status.
  • May combine two claims. 46
  • Classified as probably not a substantial claim. 30
  • The quote only partly states this claim. 1

  • May combine two claims.

    Findingcitation

    The transition from federal to state management of the Clean Water Act has little to no effect on violation rates, and this result is robust across different parallel trends assumptions and causal parameters.

    “Similarly to Grooms (2015), we find that the transition from federal to state control has little to no effect on violation rates -this result is robust across different parallel trends assumptions and different causal parameters of interest.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Grooms (2015)

      Not checked in the free run.

    • The authors state this as an assumption: In the absence of treatment, the expected outcome path is the same across all groups and all time periods.
    • The authors state this as an assumption: In the absence of treatment, the expected outcome change for a group first treated at g equals that for never-treated units.
    • The authors state this as an assumption: In the absence of treatment, the expected outcome change for a group first treated at g equals that for units not yet treated by time t.

  • May combine two claims.

    Findingnone given

    The newly proposed difference-in-differences estimator is consistent and asymptotically normal.

    “We show that this new DID estimator is consistent and asymptotically normal”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Under finite second moments, positive definite Σ, and Assumptions 2.1-2.4, when PTA 2.7 holds, the GMM estimator α_gmm is asymptotically normal and semiparametrically efficient.

    “Proposition 4.1.Assume that all random variables have finite second moments, Σ α,gmm is positive definite, and that Assumptions 2.1-2.4 hold.Then, when the parallel trends assumption 2.7 holds, we have that: (i) As n → ∞, √ n ( α gmm -α) = 1 √ n n i=1 φ gmm α (W i ) + o p (1) d → N 0, Ψ Σ -1 α,gmm Ψ -1 . (ii) The GMM estimator α gmm is semiparametrically efficient.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 2.7 imposes parallel pre-treatment trends from t = g_min -1 to T for all groups except the first-treated group.
    • The authors state this as an assumption: Assumptions 2.1-2.4 hold.
    • The authors state this as an assumption: All random variables have finite second moments and the GMM covariance matrix is positive definite.

  • May combine two claims.

    Findingreasoning

    In the data, 27 states received authorization prior to the sample period, between 1973-1975; because neither the data nor the parallel trends assumptions for Yt(0) provide information to identify the average treatment effect for these always-treated states, they are dropped from the analysis.

    “The timing of state authorization is distributed fairly evenly throughout our sample period, with the exception that 27 states received authorization prior to the sample period, between 1973-1975.16 Given that neither the data nor the parallel trends assumptions for Y t (0) provide information to identify the average treatment effect for these “always treated” states, these states are dropped from”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Assumptions 2.1-2.4 hold.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In Panel (d), the pre-treatment trend estimates are very precise zeros.

    “It is also very noticeable that pretreatment trends in Panel (d) are very precisely estimated zeros.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    TWFE and all-not-yet-treated point estimates are similar, while the never-treated comparison group implies a somewhat larger long-run effect.

    “the point estimates associated with the TWFE specification (Panel (a)) and with the estimator that uses the”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Using all not-yet-treated states as the comparison group yields a tighter simultaneous confidence interval, which suggests more efficient use of the data.

    “when using”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Corrupt states are mostly from the mid-Atlantic and southern regions, while non-corrupt states tend to appear in New England and the west.

    “Based on this measure, “corrupt” states are mostly from the mid-Atlantic and southern regions, while “non-corrupt” states tend to appear in New England and the west.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The remaining 23 states started treatment at some point, and four states are never-treated throughout the sample period.

    “Figure 6.1 highlights the year that each of the remaining 23 states started treatment, i.e., the year in which the state was authorized to administer individual NPDES permits.The bottom four states are what we call the “nevertreated” units, i.e., the states that remain unauthorized to administer individual NPDES permits through the entire sample period.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Under Assumptions 2.1-2.4 and PTA 5.1, as n→∞, the scaled difference between ATT_ny+ and ATT(g,t) has an asymptotic linear representation, and the vector of scaled differences for all t≥g converges to a normal distribution with covariance V.

    “Proposition 5.2.Assume that Assumptions 2.1-2.4 and Assumption 5.1 hold.Then, as n → ∞, √ n AT T ny+ -AT T (g, t) = 1 √ n n i=1 φ ny+ (W i ; g, t) + o p (1) . (5.6) Furthermore, √ n AT T ny+ (t ≥ g) -AT T (t ≥ g) d → N (0, V ) , (5.7) with V = E Φ ny+ (W ; t ≥ g) Φ ny+ (W ; t ≥ g) .”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 5.1: For all g,t with t≥g, the expected change in untreated outcome for group g equals the expected change in untreated outcome for units not yet treated at t.
    • The authors state this as an assumption: Assumptions 2.1-2.4 hold.

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    In the paper's application, there are 16 treatment groups and 33 time periods; the GMM associated with PTA 2.5 involves 780 moments with 195 overidentification restrictions, while the sample size of state-year pairs is 759.

    “in our application, for example, where we have 16 treatment groups and 33 time periods, the GMM involves 780 moments with 195 overidentification restrictions, whereas sample size (state-year pairs) is equal to 759.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods.

  • May combine two claims.

    Findingcitation

    The TWFE results indicate that after authorization, violation rates fell more in states with long-standing corruption than in non-corrupt states, with the gap widening as treatment time elapsed.

    “Consistent with the findings from Grooms (2015), the results suggest that states with high levels of corruption have a lower violation rate after authorization relative to non-corrupt states, and the relative drop in the violation rate appears to increase with elapsed treatment time.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Grooms (2015)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The parallel trends assumption invoked by S&A and dC&D restricts counterfactual trends after treatment and also imposes parallel pre-treatment trends.

    “We show that the PTA invoked by S&A and dC&D (i) not only restricts counterfactual trends after the treatment, but also imposes parallel pre-treatment trends”
    • Also stated in: § 2.2 The different parallel trends assumptions · plain
      “Assumption 2.5 imposes a parallel pre-trends condition across all treatment groups”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites S&A

      Not checked in the free run.

    • cites dC&D

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Different difference-in-differences procedures rely on different parallel trends assumptions and recover different causal parameters.

    “In practice, however, different DID procedures rely on different parallel trends assumptions (PTA), and recover different causal parameters.”
    • Also stated in: § 1 Introduction · plain
      “each of these papers rely on different types of parallel trends assumptions and/or propose different estimators for different causal parameters of interest.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites S&A

      Not checked in the free run.

    • cites C&S

      Not checked in the free run.

    • cites dC&D

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The parallel trends assumption invoked by S&A and dC&D imposes that every individual group not yet treated by time t can serve as a valid comparison group for earlier-treated units at time t.

    “and (ii) imposes that every individual group that is not-yet treated by time t can be used as a valid comparison group for those earlier-treated units, at time t.”
    • Also stated in: § 2.2 The different parallel trends assumptions · plain
      “allows one to use any individual group that has not yet been treated by time t (units with G s = 1, s > t) as a valid comparison group for those units already treated by time t.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites S&A

      Not checked in the free run.

    • cites dC&D

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In staggered difference-in-differences designs with multiple periods and groups, two-way fixed effects regression estimates of treatment effects can only be interpreted as weighted averages of causal effects, and some of those weights may be negative.

    “It is also worth stressing that in this more complex DID setup treatment effect estimates based on two-way fixed effects (TWFE) regression models can only be interpreted as weighted averages of causal effects, and, perhaps even more problematic, that some of these weights can be negative, see, e.g., S&A, Borusyak and Jaravel (2017), dC&D, and Goodman-Bacon (2019);”
    • Also stated in: § 2.3.1 Pitfalls of TWFE regression specifications · plain
      “when treatment effects are allowed to be heterogeneous across groups and time periods, β f e can only be interpreted as a weighted average of treatment effects, and, perhaps even more problematic, some of these weights can be negative”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites S&A

      Not checked in the free run.

    • cites Borusyak and Jaravel (2017)

      Not checked in the free run.

    • cites Goodman-Bacon (2019)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Parallel trends assumption 2.7 rules out non-parallel pre-trends for some groups and pre-treatment periods, but is still technically weaker than parallel trends assumption 2.5.

    “Importantly, and in contrast with the PTA 2.6, the PTA 2.7 does rule out non-parallel pre-trends for some groups and pre-treatment periods, 8 though, technically, it is still weaker than the PTA 2.5 as the latter completely rules out any type of non-parallel pre-trends.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    C&S derive large sample properties for their estimators and propose bootstrap procedures for simultaneous confidence bands.

    “C&S derive the large sample properties of all these aforementioned estimators and propose bootstrap procedures to construct simultaneous confidence bands for these treatment effect measures.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites C&S

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationcitation

    The parameter δ_S is by design only suitable for summarizing instantaneous treatment effects.

    “On the other hand, δ S is, by design, only suitable to summarize instantaneous treatment effects.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites dC&D

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    C&S emphasize that simultaneous inference procedures are practically important when estimating multiple parameters, because failing to account for multiple testing usually leads to misleading inference.

    “They emphasize the practical importance of using simultaneous inference procedures when one estimates multiple parameters of interest (e.g., when one estimate δ es (e) for multiple e's), as failing to account for multiple testing usually lead to misleading inference.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites C&S

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The choice of parallel trends assumption can help select the most appropriate estimator for a given parameter of interest.

    “we stress that the choice of the parallel trends assumption can also help in selecting the most appropriate estimator for a given parameter of interest.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The moment restrictions implied by parallel trends assumption 2.7 are equivalent to (2.2)-(2.5), a subset of the moment restrictions implied by parallel trends assumption 2.5.

    “Thus, by exploiting this simple but subtle observation, we can conclude that the moment restrictions implied by the PTA 2.7, (2.12)-(2.15),are equivalent (2.2)-(2.5),a subset of the moment restrictions implied by the PTA 2.5.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    For negative event times, one can use the never-treated and not-yet-treated event-study estimators as pre-trend estimators, replacing the weights with weights based on the probability of belonging to a group conditional on having at least the required number of pre-treatment periods.

    “We note that one can use δ es never (e) and/or δ es ny (e) with e < 0 as estimators of pre-trends, though, when e is negative one must replace the estimated weights w(g; e) as defined as in (2.20) with”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    PTA 2.5 implies parallel pre-treatment trends across every group, and non-zero pre-treatment event-study estimates should be interpreted as direct evidence against the identifying assumptions.

    “the PTA 2.5 implies parallel pre-treatment trends across every group, see e.g., (2.6)-(2.8),and such restrictions can be directly assessed from the data, for instance, by testing if pre-treatment event-study-type estimates of (2.23) are all equal to zero.Thus, under the PTA 2.5, non-zero pre-treatment event-study estimates should be interpreted as direct evidence against the identifying assumpt”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods.

  • May combine two claims.

    Premisereasoning

    In staggered-adoption DID designs, different parallel trends assumptions can be made and different parameters of interest can be estimated.

    “it is clear that in DID designs with staggered treatment adoption one can make different parallel trends assumptions, and can estimate different parameters of interest.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisereasoning

    When a never-treated group is not available, S&A's estimator uses only units treated at the last time period as comparison units, unlike C&S's estimator which uses not-yet-treated units.

    “Here, it is important to emphasize that, when a “never treated” group is not available, only the units treated at the last time period are used as comparison units when computing δ ge , which differs from the C&S proposed estimator (2.19) that uses not-yet treated units as comparison units.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites S&A

      Not checked in the free run.

    • cites C&S

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The difference between parallel trends assumptions 2.6 and 2.7 is that 2.6 uses never-treated units as a fixed comparison group, while 2.7 allows averages of different not-yet-treated groups as comparison.

    “The difference between the parallel trends assumptions 2.6 and 2.7 is that the former uses the “never treated” units as a fixed comparison group, whereas the latter allows one to use averages of different groups of units that are not-yet treated by time t as a comparison group.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites C&S

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Both parallel trends assumptions considered by C&S are technically weaker than the one invoked by S&A and dC&D, because they either do not restrict pre-treatment trends or impose less demanding restrictions.

    “Interestingly, both PTAs considered by C&S are, at least technically speaking, weaker than the PTA invoked by S&A and dC&D, as they either do not restrict pre-treatment trends, or, when they do, these restrictions are potentially less demanding.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites C&S

      Not checked in the free run.

    • cites S&A

      Not checked in the free run.

    • cites dC&D

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Parallel trends assumption 2.6 does not restrict pre-treatment trends across groups and does not allow later-treated units to serve as valid comparison groups for early-treated units.

    “As is evident from (2.9)-(2.11), the PTA 2.6 does not restrict pre-trends across groups, and does not presume that “later treated” units can be used as a valid comparison group for “early treated” units.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites C&S

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Parallel trends assumption 2.6 is weaker than parallel trends assumptions 2.5 and 2.7, though it requires a never-treated group.

    “In summary, from the discussion presented above, one can conclude that the PTA 2.6 does not restrict pre-trends and is weaker than the PTA 2.5 and 2.7, though it requires the existence of a “never treated” group.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    C&S consider two parallel trends assumptions: one using never-treated units as a comparison group, and one using not-yet-treated units as valid comparison groups for earlier-treated units.

    “C&S, on the other hand, considers two different PTAs, one that relies on using “never-treated” units as a comparison group, and one that uses not-yet-treated units as valid comparison groups for the earlier-treated units.”
    • Also stated in: § 2.2 The different parallel trends assumptions · plain
      “C&S consider two different PTA depending on whether a “never treated” group is available or not”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites C&S

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Under PTA 2.5, the moment restrictions form an overidentified system, so the validity of PTA 2.5 can be directly tested.

    “imposing the PTA 2.5 leads to an overidentified system of equations and, consequently, we can directly test for the validity of the PTA 2.5.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods.

  • May combine two claims.

    Interpretationreasoning

    The newly proposed difference-in-differences estimator does not restrict pre-treatment trends and uses data from all available groups.

    “Interestingly, this new DID estimator does not rely on restricting pre-treatment trends, and, at the same time, exploits data from all available groups in the given application.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    C&S show that imposing either parallel trends assumption 2.6 or 2.7 allows nonparametric point identification of all ATT(g,t) with t≥g, though each assumption yields a different estimand.

    “C&S shows that one can bypass such challenges by imposing either the PTA 2.6 or 2.7, though each assumption leads to a different estimand.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites C&S

      Not checked in the free run.

    • The authors state this as an assumption: In the absence of treatment, the expected outcome change for a group first treated at g equals that for never-treated units.
    • The authors state this as an assumption: In the absence of treatment, the expected outcome change for a group first treated at g equals that for units not yet treated by time t.

  • Classified as probably not a substantial claim.

    Interpretationcitation

    Because Grooms (2015) focuses exclusively on two-way fixed effects estimators, it is unclear what parallel trends assumption is actually being made in that analysis.

    “However, given that Grooms (2015) focuses exclusively on TWFE-type estimators, it is not clear what kind of PTA is actually being made in the analysis.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Grooms (2015)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    S&A are mainly interested in recovering the event-study-type parameter δ_es(e).

    “We conclude this subsection by noting that S&A is mainly interested in recovering the event-study-type parameter δ es (e).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites S&A

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    S&A propose an event-study estimator that is a weighted average of group-time ATT estimators and establish its large-sample properties and valid pointwise inference.

    “S&A establish the large sample properties of δ es S&A (e) and provide valid pointwise inference procedures for δ es (e).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites S&A

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    S&A propose an interaction-weighted estimator for δ_es(e).

    “More precisely, S&A propose the following interactionweighted estimator for δ es (e) (see Section 4 of S&A).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites S&A

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Under PTA 2.6, the system is just-identified, so PTA 2.6 cannot be directly tested; because it does not restrict pre-treatment trends, pre-treatment event-study estimates provide at best placebo-type evidence against PTA 2.6.

    “the conclusion is very different when one imposes the PTA 2.6: (2.9)-(2.11)suggest we have a just-identified system of equations, implying that the PTA 2.6 can not be directly tested.Indeed, as we dicussed in Section 2.2, PTA 2.6 does not restrict pretreatment trends, and, therefore, event-study estimates for pre-treatment periods provide, at best, placebo-type evidence against the PTA 2.6.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 2.6 does not restrict pre-treatment trends across groups.

  • May combine two claims.

    Interpretationnone given

    The ATT_ny+(g,t) estimand in (5.1) is suitable only for post-treatment periods t≥g; unlike (2.25) and (2.26), it cannot be fixed to analyze pre-treatment periods t<g.

    “It is worth stressing that the AT T ny+ (g, t) estimand defined in (5.1) is only suitable for post-treatment periods, i.e., for t ≥ g.Hence, in contrast to (2.25) and (2.26), we can not fix the estimand to analyze pre-treatment periods t < g.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 5.1: For all g,t with t≥g, the expected change in untreated outcome for group g equals the expected change in untreated outcome for units not yet treated at t.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Corruption is not randomly assigned, and the geographic clustering of corrupt and non-corrupt states may lead one to prefer a PTA that permits corruption-specific trends if regional trends may differ across these states.

    “In the context of our application, “corruption” is not randomly assigned and the geographic clustering of corrupt and non-corrupt states may lead us to prefer a PTA that permits corruption-specific trends if we think, for example, there may be regional trends that differ across these states.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Corruption is not randomly assigned.

  • May combine two claims.

    Interpretationreasoning

    One can build on ATT_ny+(g,t) to construct event-study estimators for δ_es(e); the natural estimator δ_es_ny+(e) is a weighted average of ATT_ny+(g,t) with weights w(g;e), and it is consistent and asymptotically normal.

    “It is straightforward to build on AT T ny+ (g, t) to construct event-study estimators for δ es (e) as defined in (2.23).Following the same steps described in Section 2.3.2, a natural estimator for δ es (e) is δ es ny+ (e) = T g=2 T t=2 1 {t -g + 1 = e} w(g; e) AT T ny+ (g, t) , where the weights w(g; e) are as defined in (2.20).By building on Theorem 5.2 and the fact that the weights admit an”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites C&S

      Not checked in the free run.

    • Rests on an assumption: The event-study weights admit an asymptotic linear representation.

  • May combine two claims.

    Interpretationreasoning

    Estimator (2.19) does not fully exploit all restrictions imposed by either PTA 2.5 or PTA 2.7, so being aware of the exact PTA allows researchers to adopt an alternative estimation procedure that fully exploits these moment restrictions and yields more efficient estimators than (2.19).

    “This is particularly important because (2.19) does not fully exploit all the restrictions imposed by either PTA.Being aware of the exact PTA invoked allows researchers to adopt an alternative estimation procedure that fully exploits all these moment restrictions, resulting in estimators that are more efficient than (2.19).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods.
    • The authors state this as an assumption: PTA 2.7 imposes parallel pre-treatment trends from t = g_min -1 to T for all groups except the first-treated group.

  • Classified as probably not a substantial claim.

    Interpretationnone given

    The main difference between PTA 2.5 and PTA 2.7 is that PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods, whereas PTA 2.7 only restricts pre-treatment trends since the time the first group is treated.

    “Recall that the main difference between PTA 2.5 and PTA 2.7 is that the former imposes parallel pre-treatment trends across all groups and all time periods, whereas the latter only restricts pre-treatment trends since the time the first group is treated.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods.
    • The authors state this as an assumption: PTA 2.7 imposes parallel pre-treatment trends from t = g_min -1 to T for all groups except the first-treated group.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    Simultaneous confidence intervals adjust for multiple testing and, asymptotically, cover the whole event-study curve with probability 1−α.

    “Simultaneous confidence intervals, on the other hand, account for this multiple testing problem, and asymptotically cover the entire event-study curve with probability 1α, where α is the significance level.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    A specific-to-general specification search that starts with stronger assumptions, tests them, and then chooses a more efficient or robust estimator based on the test is dangerous and should not be used in practice because it is based on a multiple-testing procedure, and inference that treats the final estimator as true can be severely distorted.

    “Although fairly intuitive, the aforementioned strategy is dangerous and should not be used in practice since this specification search is based on multiple-testing procedure, and, as so, inference procedures that treat the “final” estimator (or the “winner”) as “true” can be severely distorted, see, e.g.Roth (2020) for detailed discussion of this issue.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Roth (2020)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Each of the six estimators for (6.10) is explicitly linked to a specific parallel trends assumption and is designed to avoid the known problems of TWFE.

    “they are all directly and explicitly tied to a given PTA, and, by design, they bypass the potential pitfalls associated with the TWFE specification.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationnone given

    The paper proposes a simpler DID estimator that, with not-yet-treated comparison units, leaves pre-treatment trends unrestricted and draws on more groups than existing estimators.

    “We also proposed an alternative, simpler to use DID estimator that does not restrict pre-treatment trends when one wants to use”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Estimand (5.8) should not be directly compared with (2.25) and (2.26) for t<g because (5.8) measures local deviations from t-1 to t of zero pre-treatment trends, whereas (2.25) and (2.26) capture cumulative deviations from t until g-1.

    “We stress that (5.8) should not be directly compared with (2.25) and (2.26) for t < g as (5.8) measures “local deviations” (from time t -1 to t) of a zero pre-treatment trends conditions, whereas (2.25) and (2.26) capture “cumulative deviations” (from time t until g -1) of zero pre-treatment trends conditions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    In applications with multiple pre-treatment periods before the first treated group and potentially different early-period economic environments, PTA 2.7 may be favored over PTA 2.5 because early non-parallel trends become less concerning over time.

    “These differences can be meaningful in applications where data on multiple time periods before the first group of units is treated are available, and the economic environment in these “early-periods” were potentially different from the “later-periods”.In these cases, the outcome of the different groups may evolve in a non-parallel manner during “early-periods” periods, perhaps because the groups”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods.
    • The authors state this as an assumption: PTA 2.7 imposes parallel pre-treatment trends from t = g_min -1 to T for all groups except the first-treated group.

  • May combine two claims.

    Interpretationreasoning

    When g=t, the counterfactual parameter can be written as a functional of observable data, so the group-time ATT is identified by (5.1); in that case (5.1) reduces to the not-yet-treated ATT. When g differs from t, the two estimands differ.

    “First, from (5.2) and (5.4), it follows that, when g = t, α g,t (0) can be explicitly written as functionals of observable data (and not potential outcomes).As so, AT T (g, t) is identified by (5.1).Interestingly, in this case with g = t, (5.1) reduces to AT T ny (g, t), as defined in (2.26).When one moves away from the “instantaneous average treatment effects”, though, these two estimands differ.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 5.1: For all g,t with t≥g, the expected change in untreated outcome for group g equals the expected change in untreated outcome for units not yet treated at t.
    • The authors state this as an assumption: Assumptions 2.1-2.4 hold.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The estimators (5.5) and (2.27) rely on non-nested parallel trends assumptions.

    “though we stress that these two estimators rely on non-nested PTAs.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: PTA 2.6 does not restrict pre-treatment trends across groups.
    • The authors state this as an assumption: PTA 5.1: For all g,t with t≥g, the expected change in untreated outcome for group g equals the expected change in untreated outcome for units not yet treated at t.

  • May combine two claims.

    Interpretationreasoning

    When a never-treated group is available but small, inference based on (2.27) may be imprecise; this loss of efficiency is a direct consequence of not exploiting restrictions on pre-treatment trends across groups.

    “When such a group of units is available but its relative size is small, inference procedures based on (2.27) may not be as precise as one wishes.However, it is important to stress that this potential “loss of efficiency” is a direct consequence of not exploiting restrictions on pre-treatment trends across groups.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: PTA 2.6 does not restrict pre-treatment trends across groups.

  • May combine two claims.

    Interpretationreasoning

    ATT_ny+(3,4) uses data from groups G3=1, G4=1, and C=1, whereas ATT_ny(3,4) only uses data from G3=1 and C=1; therefore estimators based on (5.1) may be more precise than (2.19) because they use more data.

    “Here, we stress that AT T ny+ (3, 4) uses data from all groups, G 3 = 1, G 4 = 1, and C = 1, whereas AT T ny (3, 4) only uses data from G 3 = 1 and C = 1.Hence, one may expect that estimators based on (5.1) to be more precise than (2.19) because they utilize more data.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 5.1: For all g,t with t≥g, the expected change in untreated outcome for group g equals the expected change in untreated outcome for units not yet treated at t.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The paper formalizes six different PTAs for the application, with two natural variants of each of PTAs 2.6, 2.7, and 5.1 depending on whether different counterfactual trends between corrupt and non-corrupt states are allowed.

    “With respect to the PTA, there are two natural variants of each of the PTAs 2.6, 2.7, and 5.1 that one can invoke to highlight treatment effect heterogeneity with respect to whether a state is corrupt or not.These variants differ from each other depending on whether or not one allows for different counterfactual trends between corrupt and non-corrupt states.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods.
    • The authors state this as an assumption: PTA 2.6 does not restrict pre-treatment trends across groups.
    • The authors state this as an assumption: PTA 2.7 imposes parallel pre-treatment trends from t = g_min -1 to T for all groups except the first-treated group.
    • The authors state this as an assumption: PTA 5.1: For all g,t with t≥g, the expected change in untreated outcome for group g equals the expected change in untreated outcome for units not yet treated at t.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Assumptions 6.4-6.6 require untreated violation-rate trends to be identical for corrupt and non-corrupt states, thereby excluding corruption-specific trends.

    “Assumptions 6.4-6.6, on the other hand, assume that, in the absence of treatment, the evolution of the violation rate is the same for corrupt and noncorrupt states, i.e., it rules out corruption-specific trends.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Grooms (2015), using TWFE specifications, finds that state-level prevalence of corruption plays an important role in enforcement and compliance of environmental regulation after transitioning to state control.

    “Using TWFE specifications akin to (2.16) and (2.17), she finds that state-level prevalence of corruption plays an important role in the enforcement and compliance of environmental regulation after transitioning to state control.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Grooms (2015)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The alternative DID estimator described in Section 5 is usually less efficient than GMM but is easy to compute, does not require a never-treated group, exploits more data than (2.19), and does not explicitly restrict pre-trends because it relies on PTA 5.1.

    “Although the estimator is usually less efficient than the GMM, it is (i) easy to compute, (ii) does not require the existence of a “never-treated” group, (iii) exploits more data than (2.19), and (iv) does not explicitly restrict pre-trends as it relies on a “weaker” PTA described below.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 5.1: For all g,t with t≥g, the expected change in untreated outcome for group g equals the expected change in untreated outcome for units not yet treated at t.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    When a never-treated group is unavailable or too small, or researchers are comfortable restricting pre-treatment trends and using not-yet-treated units as comparisons, researchers can choose between PTA 2.5 and PTA 2.7, and in both cases can use the not-yet-treated ATT estimator to study policy effectiveness.

    “In such cases, researchers can then choose between the PTA 2.5 and the PTA 2.7.In both cases, though, they can use the the DID estimator AT T ny (g, t), as defined in (2.19), to study policy effectiveness.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods.
    • The authors state this as an assumption: PTA 2.7 imposes parallel pre-treatment trends from t = g_min -1 to T for all groups except the first-treated group.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The estimators in Panels (b)-(d) are intended to avoid TWFE problems and are based on explicitly stated parallel trends assumptions 2.6, 2.7, and 5.1.

    “The estimators in Panels (b), (c), and (d), on the other hand, are designed to bypass the potential pitfalls of the TWFE specification, and rely on clearly stated parallel trends assumptions (Assumptions 2.6, 2.7, and 5.1, respectively).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    PTA 5.1 imposes that the evolution of the outcome at time t among units not yet treated by time t can help identify group-time ATTs; unlike PTA 2.7, it does not require every individual not-yet-treated group to be usable as a comparison group, suggesting group-time ATTs for t≥g are nonparametrically just-identified.

    “The PTA 5.1 imposes that the evolution of the outcome at time t among those units that have not yet experienced treatment by time t can help us identify the AT T (g, t)'s.Unlike PTA 2.7, it does not impose that every individual not-yet-treated group can be used as a comparison group, which, in turn, suggests that the AT T (g, t), t ≥ g are nonparametrically just-identified.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: PTA 2.7 imposes parallel pre-treatment trends from t = g_min -1 to T for all groups except the first-treated group.
    • The authors state this as an assumption: PTA 5.1: For all g,t with t≥g, the expected change in untreated outcome for group g equals the expected change in untreated outcome for units not yet treated at t.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Assumptions 6.1-6.3 leave open the possibility that violation-rate trends differ between corrupt and non-corrupt states.

    “Assumptions 6.1-6.3)do not assume that the evolution of the violation rate is the same between corrupt and non-corrupt states.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    TWFE is a model specification and not a research design; constructing event-study estimators that explicitly rely on a particular PTA avoids the potential lack of clear interpretation associated with the TWFE specification.

    “Answering these questions is inherently hard, as TWFE is a model specification and not a “research design.”An alternative, and perhaps more constructive way of approaching this problem is to construct event-study-type estimators that explicitly rely on a particular PTA, and that, by design, avoid the potential lack of a clear interpretation associated with the TWFE specification (6.2).”
    Quote in the paperNot a substantial claim
63 of 63

  • The authors state this as an assumption.

    The observed panel data are independent and identically distributed across units.

    “Assumption 2.1 (Sampling). {Y i1 , Y i2 , . . . Y iT , D i1 , D i2 , . . . , D iT } n i=1 is independent and identically distributed (iid).”

  • The authors state this as an assumption.

    Treatment is irreversible: once a unit is treated, it remains treated in all later periods.

    “Assumption 2.2 (Staggered treatment design).For t = 2, . . ., T , D t-1 = 1 implies that D t = 1”

  • The authors state this as an assumption.

    There is no anticipatory response to treatment: for periods before a unit's treatment date, its observed outcome equals its untreated potential outcome.

    “Assumption 2.3 (No Anticipation). For all t = 1, . . . , T , g = 2, . . . , T such that t < g, E [Y it | G g = 1] = E [Y it (0)| G g = 1]”

  • The authors state this as an assumption.

    No unit is treated in the first period, and in each later period a new set of units is treated with strictly positive probability.

    “Assumption 2.4 (Overlap).P (G 1 = 1) = 0 and, for some > 0, and all g = 2, . . ., T , P (G g = 1) > .”

  • The authors state this as an assumption.

    In the absence of treatment, the expected outcome path is the same across all groups and all time periods.

    “Assumption 2.5 (Parallel trends assumption across all time periods and all groups).For all t = 2, . . ., T , all g = 2, . . ., T , E [ Y t (0) -Y t-1 (0)| G g = 1] = E [Y t (0) -Y t-1 (0)| C = 1] = E [Y t (0) -Y t-1 (0)] ,”

  • The authors state this as an assumption.

    In the absence of treatment, the expected outcome change for a group first treated at g equals that for never-treated units.

    “Assumption 2.6 (Parallel trends assumption based on “never treated” units).For all g, t = 2, . . ., T , g = 2, . . ., T , such that t ≥ g, E [Y t (0) -Y t-1 (0)| G g = 1] = E [Y t (0) -Y t-1 (0)| C = 1] .”

  • The authors state this as an assumption.

    In the absence of treatment, the expected outcome change for a group first treated at g equals that for units not yet treated by time t.

    “Assumption 2.7 (Parallel trends assumption based on “not-yet treated” units).For all g, s, t = 2, . . ., T , such that t ≥ g, s ≥ t, E [Y t (0) -Y t-1 (0)| G g = 1] = E [Y t (0) -Y t-1 (0)| D s = 0] .”

  • The authors state this as an assumption.

    PTA 2.5 imposes parallel pre-treatment trends across all groups and all time periods.

    “the former imposes parallel pre-treatment trends across all groups and all time periods”

  • The authors state this as an assumption.

    PTA 2.6 does not restrict pre-treatment trends across groups.

    “PTA 2.6 does not restrict pretreatment trends”
    “it does not impose any restriction on pre-treatment trends across groups”

  • The authors state this as an assumption.

    PTA 2.7 imposes parallel pre-treatment trends from t = g_min -1 to T for all groups except the first-treated group.

    “PTA 2.7 imposes parallel pre-treatment trends from t = g min -1 to t = T for all groups except the first-treated group (who is treated at time g min ).”

  • The authors state this as an assumption.

    PTA 5.1: For all g,t with t≥g, the expected change in untreated outcome for group g equals the expected change in untreated outcome for units not yet treated at t.

    “Assumption 5.1 (“Weaker” Parallel trends assumption based on “not-yet treated” units). For all g, t = 2, . . ., T , such that t ≥ g, E [Y t (0) -Y t-1 (0)| G g = 1] = E [Y t (0) -Y t-1 (0)| D t = 0] .”

  • The authors state this as an assumption.

    Assumptions 2.1-2.4 hold.

    “we implicitly impose Assumptions 2.1-2.4.”
    “Assume that Assumptions 2.1-2.4 hold.”

  • The authors state this as an assumption.

    PTA 6.1: For c=0,1 and all g,t with t≥g, the expected change in untreated outcome for group g conditional on corruption status c equals the expected change for never-treated units with the same corruption status.

    “Assumption 6.1 (Parallel trends assumption based on “never treated” units, with corruption-specific trends).For c = 0, 1, and all g, t = 2, . . ., T , such that t ≥ g, E [Y t (0) -Y t-1 (0)| G g = 1, Corr = c] = E [Y t (0) -Y t-1 (0)| C = 1, Corr = c] .”

  • The paper relies on this without support.

    Corruption is not randomly assigned.

    “In the context of our application, “corruption” is not randomly assigned”

  • The authors state this as an assumption.

    A stronger version of PTA 5.1 holds for both pre- and post-treatment periods.

    “which should be equal to zero under Assumptions 2.1-2.4 and a stronger version of the PTA 5.1 that holds for both pre and post-treatment periods (and not only for post-treatment periods as PTA 5.1).”

  • The authors state this as an assumption.

    All random variables have finite second moments and the GMM covariance matrix is positive definite.

    “Assume that all random variables have finite second moments, Σ α,gmm is positive definite”

  • The paper relies on this without support.

    The event-study weights admit an asymptotic linear representation.

    “By building on Theorem 5.2 and the fact that the weights admit an asymptotic linear representation”

  • The authors state this as an assumption.

    For each treatment group and corruption status, untreated outcome changes equal those for never-treated units with the same corruption status.

    “E [Y t (0) -Y t-1 (0)| G g = 1, Corr = c] = E [Y t (0) -Y t-1 (0)| C = 1, Corr = c] .”

  • The authors state this as an assumption.

    For each treatment group and corruption status, untreated outcome changes equal those for not-yet-treated units with the same corruption status, for all relevant periods.

    “Assumption 6.2 (Parallel trends assumption based on”

  • The authors state this as an assumption.

    For each treatment group and corruption status, untreated outcome changes equal those for units not yet treated at time t with the same corruption status.

    “Assumption 6.3 (”

  • The authors state this as an assumption.

    For each treatment group and corruption status, untreated outcome changes equal those for never-treated units, without conditioning on corruption status.

    “Assumption 6.4 (Parallel trends assumption based on”

  • The authors state this as an assumption.

    For each treatment group and corruption status, untreated outcome changes equal those for not-yet-treated units, without conditioning on corruption status.

    “Assumption 6.5 (Parallel trends assumption based on”

  • The authors state this as an assumption.

    For each treatment group and corruption status, untreated outcome changes equal those for units not yet treated at time t, without conditioning on corruption status.

    “Assumption 6.6 (”
23 of 23

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Marcus and Sant'Anna (2021) propose a recursive estimator that more efficiently exploits the identifying assumptions in Callaway and Sant'Anna (2021).

    “Marcus and Sant'Anna (2021) propose a recursive estimator that more efficiently exploits the identifying assumptions in Callaway and Sant'Anna (2021).”

    From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Marcus and Sant'Anna (2021)

    Section 4 (page unavailable) proposes an efficient GMM estimator that exploits the restrictions implied by the parallel-trends assumption the paper attributes to Callaway and Sant’Anna. But it describes the estimator as GMM, not recursive; Section 5’s simpler estimator is explicitly described as usually less efficient than GMM. The claim’s characterization of the proposed estimator as recursive is therefore unsupported. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Footnote reverses the post-treatment time condition

    Minor

    The footnote states the observation-based expression for ATT(g,t) when g ≥ t. Elsewhere the paper identifies ATT(g,t) for t ≥ g; when g > t, group g has not yet received treatment, so its observed outcome cannot supply the treated potential-outcome mean in that expression.

  • New plug-in estimator repeats the wrong treatment-group denominator

    Minor

    Equation (5.5) likewise sums group-g outcomes but divides by the count of group-t units. Except when g = t, that sample expression does not estimate the group-g expectation in its stated estimand (5.1).

  • Illustrative example violates the stated overlap assumption

    Minor

    The example has units first treated in periods 3 and 4, and never-treated units, but none first treated in period 2. Assumption 2.4, which the paper says it imposes when analyzing the example, requires a positive probability of first treatment in every period from 2 through T.

  • Not-yet-treated estimator divides by the wrong treatment group

    Minor

    In (2.19), the numerator sums outcomes for units first treated in period g, but the denominator counts units first treated in period t. For t > g, this is not the group-g mean specified by the population estimand (2.26); the analogous never-treated estimator (2.27) correctly divides by the count for group g.

4 of 4

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. Angrist Joshua D, Mostly harmless econometrics: An empiricist’s companion
  2. Athey Susan, 2018, Imbens
  3. Borusyak Kirill, Revisiting event study designs. Unpublished manuscript
  4. Callaway Brantly, 2020, Sant’Anna
  5. Gibbons Charles E, 2018, Juan Carlos Suárez Serrato, and Michael B. Urbancic
  6. Rambachan Ashesh, An honest approach to parallel trends. Working paper
  7. Roth Jonathan, Pre-test with caution: Event-study estimates after testing for parallel trends. Working paper

Cited by

Sorted by citations

Related

Authors
Marcus, M., & Sant’Anna, P. H. C.
ORCID
Venue
Journal of the Association of Environmental and Resource Economists
Publisher
University of Chicago Press
Published
2021-03-01
Volume
8
Issue
2
Pages
235-275
ISSN
  • 2333-5955 (ISSN-L)
  • 2333-5963
DOI
10.1086/711509
OpenAlex
W3083725402
Cited by
192
References
26
Citations by year
2026: 39 · 2025: 34 · 2024: 37 · 2023: 33 · 2022: 34 · 2021: 11 · 2020: 2 · 2019: 1 · 2018: 1
FWCI
14.03
Type
journal article
Language
en
Open access
Locations
  • Journal of the Association of Environmental and Resource Economists (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Climate Change Policy and Economics
Keywords
parallel trends assumption, staggered difference-in-differences, event study, causal identification, efficiency-robustness trade-off, Clean Water Act, environmental compliance, large sample properties

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review