Practical and Effective Approaches to Dealing With Clustered Data
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Cluster-robust standard errors (as implemented by the eponymous cluster option in Stata) can produce misleading inferences when the number of clusters G is small, even if the model is consistent and there are many observations in each cluster. Nevertheless, political scientists commonly employ this method in data sets with few clusters. The contributions of this paper are: (a) developing new and easy-to-use Stata and R packages that implement alternative uncertainty measures robust to small G, and (b) explaining and providing evidence for the advantages of these alternatives, especially cluster-adjusted t-statistics based on Ibragimov and Müller. To illustrate these advantages, we reanalyze recent work where results are based on cluster-robust standard errors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
How Much Should We Trust Differences-in-Differences Estimates?
Bertrand, M., Duflo, E., & Mullainathan, S. (2004) · The Quarterly Journal of Economics · doi:10.1162/003355304772839588 · Cited by 10,641
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Biases in dynamic models with fixed effects
Nickell, S. (1981) · Econometrica · doi:10.2307/1911408 · Cited by 8,735
Microeconometrics: Methods and Applications
Cameron, A. C., & Trivedi, P. K. (2005) · doi:10.1017/cbo9780511811241 · Cited by 8,092
What To Do (and Not to Do) with Time-Series Cross-Section Data
Beck, N., & Katz, J. N. (1995) · American Political Science Review · doi:10.2307/2082979 · Cited by 6,840
Understanding Interaction Models: Improving Empirical Analyses
Brambor, T., Clark, W. R., & Golder, M. (2006) · Political Analysis · doi:10.1093/pan/mpi014 · Cited by 6,223
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
A Practitioner’s Guide to Cluster-Robust Inference
Colin Cameron, A., & Miller, D. L. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.317 · Cited by 5,037
Vaart, A. W. V. D. (1998) · doi:10.1017/cbo9780511802256 · Cited by 4,159
A Note on Robust Variance Estimation for Cluster‐Correlated Data
Williams, R. L. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00645.x · Cited by 2,501 · cited 1×
“…of papers have emphasized their ability to obtain accurate measures of uncertainty (and appropriately sized statistical significance tests) in a wide variety of scenarios (Liang and Zeger 1993; Donner 1998; Williams 2000; Klar and Donner 2001; Kezdi 2004; Bertrand, Duflo and Mullainathan 2004).”
BackgroundGeneralized Estimating Equations
Hardin, J. W., & Hilbe, J. M. (2002) · doi:10.1201/9781420035285 · Cited by 1,766
Inference with Difference-in-Differences and Other Panel Data
Donald, S. G., & Lang, K. (2007) · Review of Economics and Statistics · doi:10.1162/rest.89.2.221 · Cited by 1,324
Panel Data Econometrics in R : The plm Package
Croissant, Y., & Millo, G. (2008) · Journal of Statistical Software · doi:10.18637/jss.v027.i02 · Cited by 1,243 · cited 1×
“Under these circumstances, we use the plm package in R (Croissant and Millo 2008) to add a fixed effect for g to models before correcting the SEs for clustering.”
MethodAsymptotic properties of a robust variance matrix estimator for panel data when is large
Hansen, C. B. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2006.10.009 · Cited by 293
Randomization Tests Under an Approximate Symmetry Assumption
Canay, I. A., Romano, J. P., & Shaikh, A. M. (2017) · Econometrica · doi:10.3982/ecta13081 · Cited by 125
Cited by
Heisig, J. P., & Schaeffer, M. (2019) · European Sociological Review · doi:10.1093/esr/jcy053 · Cited by 552
Cluster-robust inference: A guide to empirical practice
MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2022.04.001 · Cited by 407
“and Miller (2015), MacKinnon (2019), Esarey and Menger (2019), and MacKinnon and Webb (2020a). Conley, Gonçalves and Hansen (2018) surveys a broader class of methods for various types of dependent data.”
BackgroundHousehold electrification and indoor air pollution
Barron, M., & Torero, M. (2017) · Journal of Environmental Economics and Management · doi:10.1016/j.jeem.2017.07.007 · Cited by 180
What drives the erasure of protected areas? Evidence from across the Brazilian Amazon
Keles, D., Delacote, P., Pfaff, A., Qin, S., & Mascia, M. B. (2020) · Ecological Economics · doi:10.1016/j.ecolecon.2020.106733 · Cited by 43
“We have 9 clusters, not enough to guarantee consistent estimates of standard errors (Cameron and Miller, 2015), yet we cannot rely a on non-parametric bootstrap (Esarey and Menger, 2018) because we don’t have enough variations within each cluster. add less damage.”
Background
Metadata
- Authors
- Esarey, J., & Menger, A.
- Venue
- Political Science Research and Methods
- Publisher
- Cambridge University Press (CUP)
- Published
- 2018-01-19
- Volume
- 7
- Issue
- 3
- Pages
- 541-559
- ISSN
- 2049-8470 (ISSN-L)
- 2049-8489
- OpenAlex
- W2784542055
- Cited by
- 141
- References
- 15
- FWCI
- 21.07
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.