The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Randomization inference for difference-in-differences with few treated clusters
Open accessAbstract
Inference using difference-in-differences with clustered data requires care. Previous research has shown that, when there are few treated clusters, t-tests based on cluster-robust variance estimators (CRVEs) severely overreject, and different variants of the wild cluster bootstrap can either overreject or underreject dramatically. We study two randomization inference (RI) procedures. A procedure based on estimated coefficients may be unreliable when clusters are heterogeneous. A procedure based on t-statistics typically performs better (although by no means perfectly) under the null, but at the cost of some power loss. An empirical example demonstrates that alternative procedures can yield dramatically different inferences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Practitioner’s Guide to Cluster-Robust Inference
Colin Cameron, A., & Miller, D. L. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.317 · Cited by 5,037
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Inference with “Difference in Differences” with a Small Number of Policy Changes
Conley, T. G., & Taber, C. R. (2011) · Review of Economics and Statistics · doi:10.1162/rest_a_00049 · Cited by 633
Inference with dependent data using cluster covariance estimators
Bester, C. A., Conley, T. G., & Hansen, C. B. (2011) · Journal of Econometrics · doi:10.1016/j.jeconom.2011.01.007 · Cited by 258
Inference in Differences-in-Differences with Few Treated Groups and Heteroskedasticity
Ferman, B., & Pinto, C. (2019) · The Review of Economics and Statistics · doi:10.1162/rest_a_00759 · Cited by 178
Randomization Tests Under an Approximate Symmetry Assumption
Canay, I. A., Romano, J. P., & Shaikh, A. M. (2017) · Econometrica · doi:10.3982/ecta13081 · Cited by 125
Inference with few heterogeneous clusters
Ibragimov, R., & Müller, U. K. (2016) · Review of Economics and Statistics · doi:10.1162/rest_a_00545 · Cited by 99
Research works printed but not linked: 18
- Bailey, 2010, “Momma’s got the pill”: How Anthony Comstock and Griswold v. Connecticut shaped US childbearing, Amer. Econ. Rev., 100, 98, 10.1257/aer.100.1.98 · DOI
- Barrios, 2012, Clustering, spatial correlations, and randomization inference, J. Amer. Statist. Assoc., 107, 578, 10.1080/01621459.2012.682524 · DOI
- Bell, 2002, Bias reduction in standard errors for linear regression with multi-stage samples, Surv. Methodol., 28, 169
- Cameron, 2008, Bootstrap-based improvements for inference with clustered errors, Rev. Econ. Stat., 90, 414, 10.1162/rest.90.3.414 · DOI · Paper page
- Carter, 2017, Asymptotic behavior of a t test robust to cluster heterogeneity, Rev. Econ. Stat., 99, 698, 10.1162/REST_a_00639 · DOI
- DiCiccio, 2017, Robust permutation tests for correlation and regression coefficients, J. Amer. Statist. Assoc., 112, 1211, 10.1080/01621459.2016.1202117 · DOI
- Djogbenou, 2019, Asymptotic theory and wild bootstrap inference with clustered errors, J. Econometrics, 212, 393, 10.1016/j.jeconom.2019.04.035 · DOI
- Dufour, 2006, Monte Carlo tests with nuisance parameters: A general approach to finite-sample inference and nonstandard asymptotics, J. Econometrics, 133, 443, 10.1016/j.jeconom.2005.06.007 · DOI · Paper page
- Hagemann, 2019, Placebo inference on treatment effects when the number of clusters is small, J. Econometrics, 212, 190, 10.1016/j.jeconom.2019.04.011 · DOI
- Hess, 2017, Randomization inference with Stata: A guide and software, Stata J., 17, 630, 10.1177/1536867X1701700306 · DOI · Paper page
- Imbens, 2016, Robust standard errors in small samples: Some practical advice, Rev. Econ. Stat., 98, 701, 10.1162/REST_a_00552 · DOI · Paper page
- Liang, 1986, Longitudinal data analysis using generalized linear models, Biometrika, 73, 13, 10.1093/biomet/73.1.13 · DOI · Paper page
- MacKinnon, 2017, Wild bootstrap inference for wildly different cluster sizes, J. Appl. Econometrics, 32, 233, 10.1002/jae.2508 · DOI · Paper page
- MacKinnon, 2018, The wild bootstrap for few (treated) clusters, Econom. J., 21, 114, 10.1111/ectj.12107 · DOI · Paper page
- Romano, 1990, On the behavior of randomization tests without a group invariance assumption, J. Amer. Statist. Assoc., 85, 686, 10.1080/01621459.1990.10474928 · DOI · Paper page
- Rosenbaum, 1996, Observational studies and nonrandomized experiments, Handbook of Statist., 13, 181, 10.1016/S0169-7161(96)13008-X · DOI
- Yates, 1984, Tests of significance for 2 × 2 contingency tables, J. Roy. Statist. Soc. Ser. A, 147, 426, 10.2307/2981577 · DOI · Paper page
- Young, 2019, Channelling Fisher: Randomization tests and the statistical insignificance of seemingly significant experimental results, Q. J. Econ., 134, 557, 10.1093/qje/qjy029 · DOI · Paper page
Cited by
Difference-in-Differences with multiple time periods
Callaway, B., & Sant’Anna, P. H. C. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.12.001 · Cited by 9,126
The triple difference estimator
Olden, A., & Møen, J. (2022) · The Econometrics Journal · doi:10.1093/ectj/utac010 · Cited by 432
Cluster-robust inference: A guide to empirical practice
MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2022.04.001 · Cited by 407
Self-citationThe wild bootstrap for few (treated) clusters
MacKinnon, J. G., & Webb, M. D. (2018) · The Econometrics Journal · doi:10.1111/ectj.12107 · Cited by 317
Self-citationInference in Differences-in-Differences with Few Treated Groups and Heteroskedasticity
Ferman, B., & Pinto, C. (2019) · The Review of Economics and Statistics · doi:10.1162/rest_a_00759 · Cited by 178
Metadata
- Authors
- MacKinnon, J. G., & Webb, M. D.
- Venue
- Journal of Econometrics
- Publisher
- Elsevier BV
- Published
- 2020-10
- Volume
- 218
- Issue
- 2
- Pages
- 435-450
- ISSN
- 0304-4076 (ISSN-L)
- OpenAlex
- W2258423801
- Cited by
- 167
- References
- 7
- FWCI
- 12.09
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- Funders
- Social Sciences and Humanities Research Council of Canada
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Randomization inference for difference-in-differences with few treated clusters. https://app.cochat.ai/papers/10.1016/j.jeconom.2020.04.024
- The paper
- MacKinnon, J. G., & Webb, M. D. (2020). Randomization inference for difference-in-differences with few treated clusters. Journal of Econometrics. https://doi.org/10.1016/j.jeconom.2020.04.024
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
