The Wild Bootstrap with a “Small” Number of “Large” Clusters
The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
This paper studies the wild bootstrap–based test proposed in Cameron, Gelbach, and Miller (2008). Existing analyses of its properties require that number of clusters is “large.” In an asymptotic framework in which the number of clusters is “small,” we provide conditions under which an unstudentized version of the test is valid. These conditions include homogeneity-like restrictions on the distribution of covariates. We further establish that a studentized version of the test may only overreject the null hypothesis by a “small” amount that decreases exponentially with the number of clusters. We obtain a qualitatively similar result for “score” bootstrap-based tests, which permit testing in nonlinear models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, despite strong simulation performance, the wild cluster bootstrap technically invokes an assumption of a balanced proportion of treated units among clusters, which may not be plausible in many DiD applications where the treatment is assigned at the cluster level (see Canay et al. [117]).”
From Difference-in-Differences for Health Policy and Practice: A Review of Modern Methods (2025) · cites this paper as [117]
In the citing paper
“Canay et al. (2021) show that the reliability of these bootstrap procedures depends on imposing certain homogeneity conditions on treatment effects, as well as the type of bootstrap weights one uses and the estimation method adopted (e.g., restricted vs. unrestricted OLS).”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Canay, Santos and Shaikh (2021)
In the citing paper
“These restrictions are commonly violated when one uses TWFE regressions with cluster-specific and time fixed effects like (19) or when treatment effects are allowed to be heterogeneous across clusters”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Canay et al. (2021)
Citations
Cites
1 retracted
Retracted: Growing up in a Recession
Giuliano, P., & Spilimbergo, A. (2014) · The Review of Economic Studies · doi:10.1093/restud/rdt040 · Cited by 480
RetractedLarge Sample Properties of Generalized Method of Moments Estimators
Hansen, L. P. (1982) · Econometrica · doi:10.2307/1912775 · Cited by 13,467
How Much Should We Trust Differences-in-Differences Estimates?
Bertrand, M., Duflo, E., & Mullainathan, S. (2004) · The Quarterly Journal of Economics · doi:10.1162/003355304772839588 · Cited by 10,641
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
Weak Convergence and Empirical Processes
van der Vaart, A. W., & Wellner, J. A. (1996) · Springer Series in Statistics · doi:10.1007/978-1-4757-2545-2 · Cited by 5,014
Bootstrap-Based Improvements for Inference With Clustered Errors
Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2008) · Review of Economics and Statistics · doi:10.1162/rest.90.3.414 · Cited by 4,417
Random group effects and the precision of regression estimates
Moulton, B. R. (1986) · Journal of Econometrics · doi:10.1016/0304-4076(86)90021-7 · Cited by 1,626
Inference with Difference-in-Differences and Other Panel Data
Donald, S. G., & Lang, K. (2007) · Review of Economics and Statistics · doi:10.1162/rest.89.2.221 · Cited by 1,324
Testing Statistical Hypotheses
Neath, A. A. (2006) · Journal of the American Statistical Association · doi:10.1198/jasa.2006.s100 · Cited by 948
Bootstrap and Wild Bootstrap for High Dimensional Linear Models
Mammen, E. (1993) · The Annals of Statistics · doi:10.1214/aos/1176349025 · Cited by 889
Cluster-Sample Methods in Applied Econometrics
Wooldridge, J. M. (2003) · American Economic Review · doi:10.1257/000282803321946930 · Cited by 867
Bootstrap Procedures under some Non-I.I.D. Models
Liu, R. Y. (1988) · The Annals of Statistics · doi:10.1214/aos/1176351062 · Cited by 640
The wild bootstrap, tamed at last
Davidson, R., & Flachaire, E. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2008.08.003 · Cited by 506
The Consequences of Radical Reform: The French Revolution
Acemoglu, D., Cantoni, D., Johnson, S., & Robinson, J. A. (2011) · American Economic Review · doi:10.1257/aer.101.7.3286 · Cited by 460
Wild Bootstrap Inference for Wildly Different Cluster Sizes
MacKinnon, J. G., & Webb, M. D. (2017) · Journal of Applied Econometrics · doi:10.1002/jae.2508 · Cited by 454
t-Statistic Based Correlation and Heterogeneity Robust Inference
Ibragimov, R., & Müller, U. K. (2010) · Journal of Business & Economic Statistics · doi:10.1198/jbes.2009.08046 · Cited by 299
The Institutional Causes of China's Great Famine, 1959–1961
Meng, X., Qian, N., & Yared, P. (2015) · The Review of Economic Studies · doi:10.1093/restud/rdv016 · Cited by 274
Inference with dependent data using cluster covariance estimators
Bester, C. A., Conley, T. G., & Hansen, C. B. (2011) · Journal of Econometrics · doi:10.1016/j.jeconom.2011.01.007 · Cited by 258
THE SIZE DISTORTION OF BOOTSTRAP TESTS
Davidson, R., & MacKinnon, J. G. (1999) · Econometric Theory · doi:10.1017/s0266466699153040 · Cited by 206
A score based approach to wild bootstrap inference
Kline, P., & Santos, A. (2012) · Journal of Econometric Methods · doi:10.1515/2156-6674.1006 · Cited by 161
Self-citationKosfeld, M., & Rustagi, D. (2015) · American Economic Review · doi:10.1257/aer.20120700 · Cited by 151
Randomization Tests Under an Approximate Symmetry Assumption
Canay, I. A., Romano, J. P., & Shaikh, A. M. (2017) · Econometrica · doi:10.3982/ecta13081 · Cited by 125 · cited 1×
“This distinction prevents the application of results in Canay et al. (2017). In fact, sn and ŝn do not even tend in distribution to the same limit.”
BackgroundSelf-citationThursby, J. G., Amemiya, T., Fomby, T. B., Hill, R. C., & Johnson, S. R. (1987) · Journal of the American Statistical Association · doi:10.2307/2288830 · Cited by 113
Generating Correlation Matrices
Marsaglia, G., & Olkin, I. (1984) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0905034 · Cited by 110
Asymptotic theory and wild bootstrap inference with clustered errors
Djogbenou, A. A., MacKinnon, J. G., & Nielsen, M. Ø. (2019) · Journal of Econometrics · doi:10.1016/j.jeconom.2019.04.035 · Cited by 104
Inference with few heterogeneous clusters
Ibragimov, R., & Müller, U. K. (2016) · Review of Economics and Statistics · doi:10.1162/rest_a_00545 · Cited by 99 · cited 3×
“It follows that when these conditions are implausible and there are few clusters, researchers may wish to consider methods that do not impose such restrictions, such as Ibragimov and Müller (2010) and Canay et al.”
BackgroundResultInfluentialAsymptotic Behavior of a t Test Robust to Cluster Heterogeneity
Steigerwald, D. G. (2013) · Cited by 22
Bootstrap and Asymptotic Inference with Multiway ClusteringPreprint
MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2017) · RePEc: Research Papers in Economics · doi:10.22004/ag.econ.274712 · Cited by 3
Cited by
What's trending in difference-in-differences? A synthesis of the recent econometrics literature
Roth, J., Sant’Anna, P. H. C., Bilinski, A., & Poe, J. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2023.03.008 · Cited by 2,027
“Importantly, Canay et al. (2021) show that the reliability of these bootstrap procedures depends on imposing certain homogeneity conditions on treatment effects, as well as the type of bootstrap weights one uses and the estimation method adopted (e.g., restricted vs. unrestricted OLS).”
InfluentialCluster-robust inference: A guide to empirical practice
MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2022.04.001 · Cited by 407
“The results of Canay et al. (2021) and MacKinnon and Webb (2020b) strongly suggest that homogeneity across clusters is more important for RI than for the WCR bootstrap.”
BackgroundConducting Research in Marketing with Quasi-Experiments
Goldfarb, A., Tucker, C., & Wang, Y. (2022) · Journal of Marketing · doi:10.1177/00222429221082977 · Cited by 185
Reworking wild bootstrap‐based inference for clustered errors
Webb, M. D. (2023) · Canadian Journal of Economics/Revue canadienne d économique · doi:10.1111/caje.12661 · Cited by 226
Jordà, Ò., & Taylor, A. M. (2025) · Journal of Economic Literature · doi:10.1257/jel.20241521 · Cited by 217
Metadata
- Authors
- Canay, I. A., Santos, A., & Shaikh, A. M.
- Affiliations
- Northwestern University (US)
- University of California, Los Angeles (US)
- University of Chicago (US)
- ORCID
- Canay, Ivan A.
- Venue
- The Review of Economics and Statistics
- Publisher
- MIT Press - Journals
- Published
- 2021-05
- Volume
- 103
- Issue
- 2
- Pages
- 346-363
- ISSN
- 0034-6535 (ISSN-L)
- 1530-9142
- OpenAlex
- W4206904707
- Cited by
- 104
- References
- 28
- Citations by year
- 2026: 20 · 2025: 26 · 2024: 17 · 2023: 19 · 2022: 8 · 2021: 7
- FWCI
- 4.10
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Review of Economics and Statistics (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Keywords
- wild bootstrap, small number of clusters, cluster-robust inference, nonlinear models, panel data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.