Inference with “Difference in Differences” with a Small Number of Policy Changes
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
In difference-in-differences applications, identification of the key parameter often arises from changes in policy by a small number of groups. In contrast, typical inference assumes that the number of groups changing policy is large. We present an alternative inference approach for a small (finite) number of policy changers, using information from a large sample of nonchanging groups. Treatment effect point estimators are not consistent, but we can consistently estimate their asymptotic distribution under any point null hypothesis about the treatment. Thus, treatment point estimators can be used as test statistics, and confidence intervals can be constructed using test statistic inversion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Nevertheless, the assumption that all treated groups have the same distribution of errors is still strong, and will often be violated if either there is heterogeneity in treatment effects or in cluster sizes.”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Conley and Taber (2011)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The key idea in Conley and Taber (2011) is that if we assume that the errors in treated states come from the same distribution as in control states, then we can learn the distribution of errors from the large number of control states and use that to construct standard errors.”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Conley and Taber (2011)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Conley and Taber proposed an approach to estimate the distribution of errors in the treated clusters using that from the untreated clusters [40]. This approach may be powerful when there are only a few treated clusters but many untreated ones from which to estimate the error distribution.”
From Difference-in-Differences for Health Policy and Practice: A Review of Modern Methods (2025) · cites this paper as [40]
The cited source says what the claim attributes to it.
In the citing paper
“Conley and Taber (2011) introduce an alternative approach for inference that is able to relax the strong assumption of Gaussian errors in settings where there are many control clusters ( J 0 large) but few treated clusters ( J 1 small).”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Conley and Taber (2011)
The cited source says what the claim attributes to it.
In the citing paper
“A key advantage of this approach is that the distribution of errors is not assumed to be Gaussian, but rather is learned from the data.”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Conley and Taber (2011)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Filders, R. (1992) · International Journal of Forecasting · doi:10.1016/0169-2070(92)90130-2 · Cited by 16,165
How Much Should We Trust Differences-in-Differences Estimates?
Bertrand, M., Duflo, E., & Mullainathan, S. (2004) · The Quarterly Journal of Economics · doi:10.1162/003355304772839588 · Cited by 10,641
Abadie, A., Diamond, A., & Hainmueller, J. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.ap08746 · Cited by 5,513
Earnings Losses of Displaced Workers
Jacobson, L. S., LaLonde, R. J., & Sullivan, D. G. (1992) · doi:10.17848/wp92-11 · Cited by 2,895
GMM Estimation with Cross Sectional Dependence
Conley, T. G. (1999) · Journal of Econometrics · doi:10.1016/s0304-4076(98)00084-0 · Cited by 2,685
Self-citationAn Illustration of a Pitfall in Estimating the Effects of Aggregate Variables on Micro Units
Moulton, B. R. (1990) · The Review of Economics and Statistics · doi:10.2307/2109724 · Cited by 2,630
Analysis of Covariance with Qualitative Data
Chamberlain, G. (1980) · The Review of Economic Studies · doi:10.2307/2297110 · Cited by 2,629
Card, D., & Krueger, A. B. (2000) · American Economic Review · doi:10.1257/aer.90.5.1397 · Cited by 2,206
Natural and Quasi- Experiments in Economics
Meyer, B. (1994) · Journal of Business and Economic Statistics · doi:10.2307/1392369 · Cited by 1,792
The Impact of the Mariel Boatlift on the Miami Labor Market
Card, D. (1990) · ILR Review · doi:10.1177/001979399004300205 · Cited by 1,704
Research works printed but not linked: 1
- Bertrand Marianne, 2002, NBER working paper, 8841
Cited by
Items 601–625 of 625
Contrasting Effects of EU and EMU Membership on FDI : A Synthetic Difference‐in‐Differences Approach
Ryan, M., & Tanaka, A. (2026) · Review of International Economics · doi:10.1111/roie.70090
Recognition rather than buffers. How a supervisor's first climate action affects bank provisioningPreprint
Zubir, A. S. M., Hashim, H. A., Chuah, S.-F., Ariff, A. M., & Salleh, Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7519202
Cushioned but hollowed out: coal’s end and the demographic legacy of Spain’s mining communities
González, F. A. I. (2026) · The Extractive Industries and Society · doi:10.1016/j.exis.2026.102061
Beyond Mortality: The Effect of Triplicate Prescription Programs on Birth Outcomes
Jun, D., & Kim, B. (2026) · The B E Journal of Economic Analysis & Policy · doi:10.1515/bejeap-2026-0036 · Cited by 1
“20 Because only five states adopted triplicate programs (Conley and Taber 2011), we supplement the asymptotic standard errors above with a permutation test (MacKinnon and Webb 2020) that does not rely on asymptotic approximations (see Online Appendix E for details).”
The Missing Cohort: Smaller Classes after the Zika Announcement in BrazilPreprint
Pieri, R. G. D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7590459
Lending Through the Fire: Credit Expansion and Rising Bank Risk After the 2025 California Wildfires
Crowley, D., Guo, L., & Han, K. (2026) · Risks · doi:10.3390/risks14100234
Beyond file-by-file insularity: Measuring cumulative EU regulatory burden on the Greek economy
Barone, E., & Katsikas, E. (2026) · Research in Economics · doi:10.1016/j.rie.2026.101188
Guo, Z., & Lu, H. (2026) · Ecological Chemistry and Engineering S · doi:10.2478/eces-2026-0020
“Conley and Taber [52] caution that standard DID inference may be unreliable when only a small number of groups experience policy changes.”
Economic Headroom as a Compound TreatmentPreprint
Xiang, G. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7594638
Air cargo liberalization under joint production: Reassessing the anchorage hub
Zhang, Y. (2026) · Transport Economics and Management · doi:10.1016/j.team.2026.10.003
Zhang, S., Zhao, L., & Liu, Y. (2026) · Sustainability · doi:10.3390/su182010425
Xu, F., & Lei, Y. (2026) · Economic Analysis and Policy · doi:10.1016/j.eap.2026.10.044
Cash Transfers and Female Labour Supply: Evidence from Six Natural Experiments in IndiaPreprint
Raghunathan, R., Shah, P., & Parle, P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7607159
Migkos, S. P., Katarachia, A., Farmaki, P., & Tranoulidis, A. (2026) · Economies · doi:10.3390/economies14100485
Import Intensity and Currency Invoicing Choice: Evidence from the Brexit Referendum
Drissi, R. (2026) · Economies · doi:10.3390/economies14100486
The Handshake Premium: How Markets Priced the Abraham AccordsPreprint
Ichev, R., & Spruk, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7643253
Strict liability and moral hazard: Evidence from New York’s Scaffold law
Geddes, R. R., Park, Y. G., & Zhang, K. (2026) · International Review of Law and Economics · doi:10.1016/j.irle.2026.106361
Do Academic Credentials Still Predict Professional Competence After Generative AI?Preprint
Bamasoud, D. M., Alqahtani, M. A., & Mohammad, R. (2026) · Research Square · doi:10.21203/rs.3.rs-11163181/v1
Liquidity Shocks and Drug Sales: Evidence From India
Chakrabarti, A. S., Chatterjee, C., & Gupta, S. (2026) · Journal of Economics & Management Strategy · doi:10.1111/jems.70048
Two-cluster Difference-in-Differences via Maximally Detectable Placebo EffectPreprint
Ramos, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7645879
Danso, F. K., Mkhize, M. V., Prempeh, K. B., & Owusu, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7656759
Sectoral reallocation without tariff effects: evidence from the Turkey–morocco free trade agreement
MOUGHAMIR, B., & Mansouri, Z. (2026) · SN Business & Economics · doi:10.1007/s43546-026-01523-9
“…randomization p -value is Adding the placebo indicator rather than relabelling the treatment keeps the estimated specification identical across draws, in the spirit of Conley and Taber ( 2011); the relabelling variant, in which 𝐷𝐷 (𝑗𝑗 ′ ) replaces fta_turkey , gives the same…”
InfluentialBank Soundness in Conflict Zones: Global Evidence with a Focus on MENAPreprint
Ashour, T. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7675066
D'Alessio, S. J., & Stolzenberg, L. (2026) · American Journal of Criminal Justice · doi:10.1007/s12103-026-09962-8
Metadata
- Authors
- Conley, T. G., & Taber, C. R.
- Affiliations
- University of Chicago (US)
- University of Wisconsin–Madison (US)
- ORCID
- Conley, Timothy G.
- Venue
- Review of Economics and Statistics
- Publisher
- MIT Press
- Published
- 2011-02
- Volume
- 93
- Issue
- 1
- Pages
- 113-125
- ISSN
- 0034-6535 (ISSN-L)
- 1530-9142
- OpenAlex
- W2097636528
- Cited by
- 633
- References
- 44
- Citations by year
- 2026: 79 · 2025: 36 · 2024: 45 · 2023: 39 · 2022: 52 · 2021: 50 · 2020: 39 · 2019: 50 · 2018: 49 · 2017: 40 · 2016: 36 · 2015: 36 · 2014: 18 · 2013: 24 · 2012: 14
- FWCI
- 14.89
- Type
- journal article
- Language
- en
- Open access
- green
- www.nber.org
- Locations
- The Review of Economics and Statistics (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- difference-in-differences, treatment effect estimation, asymptotic distribution, point null hypothesis, confidence intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.