Inference with “Difference in Differences” with a Small Number of Policy Changes
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
In difference-in-differences applications, identification of the key parameter often arises from changes in policy by a small number of groups. In contrast, typical inference assumes that the number of groups changing policy is large. We present an alternative inference approach for a small (finite) number of policy changers, using information from a large sample of nonchanging groups. Treatment effect point estimators are not consistent, but we can consistently estimate their asymptotic distribution under any point null hypothesis about the treatment. Thus, treatment point estimators can be used as test statistics, and confidence intervals can be constructed using test statistic inversion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Nevertheless, the assumption that all treated groups have the same distribution of errors is still strong, and will often be violated if either there is heterogeneity in treatment effects or in cluster sizes.”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Conley and Taber (2011)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The key idea in Conley and Taber (2011) is that if we assume that the errors in treated states come from the same distribution as in control states, then we can learn the distribution of errors from the large number of control states and use that to construct standard errors.”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Conley and Taber (2011)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Conley and Taber proposed an approach to estimate the distribution of errors in the treated clusters using that from the untreated clusters [40]. This approach may be powerful when there are only a few treated clusters but many untreated ones from which to estimate the error distribution.”
From Difference-in-Differences for Health Policy and Practice: A Review of Modern Methods (2025) · cites this paper as [40]
The cited source says what the claim attributes to it.
In the citing paper
“Conley and Taber (2011) introduce an alternative approach for inference that is able to relax the strong assumption of Gaussian errors in settings where there are many control clusters ( J 0 large) but few treated clusters ( J 1 small).”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Conley and Taber (2011)
The cited source says what the claim attributes to it.
In the citing paper
“A key advantage of this approach is that the distribution of errors is not assumed to be Gaussian, but rather is learned from the data.”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Conley and Taber (2011)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Filders, R. (1992) · International Journal of Forecasting · doi:10.1016/0169-2070(92)90130-2 · Cited by 16,165
How Much Should We Trust Differences-in-Differences Estimates?
Bertrand, M., Duflo, E., & Mullainathan, S. (2004) · The Quarterly Journal of Economics · doi:10.1162/003355304772839588 · Cited by 10,641
Abadie, A., Diamond, A., & Hainmueller, J. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.ap08746 · Cited by 5,513
Earnings Losses of Displaced Workers
Jacobson, L. S., LaLonde, R. J., & Sullivan, D. G. (1992) · doi:10.17848/wp92-11 · Cited by 2,895
GMM Estimation with Cross Sectional Dependence
Conley, T. G. (1999) · Journal of Econometrics · doi:10.1016/s0304-4076(98)00084-0 · Cited by 2,685
Self-citationAn Illustration of a Pitfall in Estimating the Effects of Aggregate Variables on Micro Units
Moulton, B. R. (1990) · The Review of Economics and Statistics · doi:10.2307/2109724 · Cited by 2,630
Analysis of Covariance with Qualitative Data
Chamberlain, G. (1980) · The Review of Economic Studies · doi:10.2307/2297110 · Cited by 2,629
Card, D., & Krueger, A. B. (2000) · American Economic Review · doi:10.1257/aer.90.5.1397 · Cited by 2,206
Natural and Quasi- Experiments in Economics
Meyer, B. (1994) · Journal of Business and Economic Statistics · doi:10.2307/1392369 · Cited by 1,792
The Impact of the Mariel Boatlift on the Miami Labor Market
Card, D. (1990) · ILR Review · doi:10.1177/001979399004300205 · Cited by 1,704
Research works printed but not linked: 1
- Bertrand Marianne, 2002, NBER working paper, 8841
Cited by
Difference-in-Differences with multiple time periods
Callaway, B., & Sant’Anna, P. H. C. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.12.001 · Cited by 9,126
Abadie, A., Diamond, A., & Hainmueller, J. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.ap08746 · Cited by 5,513
A Practitioner’s Guide to Cluster-Robust Inference
Colin Cameron, A., & Miller, D. L. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.317 · Cited by 5,037
What's trending in difference-in-differences? A synthesis of the recent econometrics literature
Roth, J., Sant’Anna, P. H. C., Bilinski, A., & Poe, J. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2023.03.008 · Cited by 2,027
“The key idea in Conley and Taber (2011) is that if we assume that the errors in treated states come from the same distribution as in control states, then we can learn the distribution of errors from the large number of control states and use that to construct standard errors.”
Designing Difference in Difference Studies: Best Practices for Public Health Policy Research
Wing, C., Simon, K., & Bello-Gomez, R. A. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-013507 · Cited by 1,470
Metadata
- Authors
- Conley, T. G., & Taber, C. R.
- Affiliations
- University of Chicago (US)
- University of Wisconsin–Madison (US)
- ORCID
- Conley, Timothy G.
- Venue
- Review of Economics and Statistics
- Publisher
- MIT Press
- Published
- 2011-02
- Volume
- 93
- Issue
- 1
- Pages
- 113-125
- ISSN
- 0034-6535 (ISSN-L)
- 1530-9142
- OpenAlex
- W2097636528
- Cited by
- 633
- References
- 44
- Citations by year
- 2026: 79 · 2025: 36 · 2024: 45 · 2023: 39 · 2022: 52 · 2021: 50 · 2020: 39 · 2019: 50 · 2018: 49 · 2017: 40 · 2016: 36 · 2015: 36 · 2014: 18 · 2013: 24 · 2012: 14
- FWCI
- 14.89
- Type
- journal article
- Language
- en
- Open access
- green
- www.nber.org
- Locations
- The Review of Economics and Statistics (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- difference-in-differences, treatment effect estimation, asymptotic distribution, point null hypothesis, confidence intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Inference with “Difference in Differences” with a Small Number of Policy Changes. https://app.cochat.ai/papers/10.1162/rest_a_00049
- The paper
- Conley, T. G., & Taber, C. R. (2011). Inference with “Difference in Differences” with a Small Number of Policy Changes. Review of Economics and Statistics. https://doi.org/10.1162/rest_a_00049
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.