The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Robust Standard Errors in Small Samples: Some Practical Advice
Abstract
We study the properties of heteroskedasticity-robust confidence intervals for regression parameters. We show that confidence intervals based on a degrees-of-freedom correction suggested by Bell and McCaffrey (2002) are a natural extension of a principled approach to the Behrens-Fisher problem. We suggest a further improvement for the case with clustering. We show that these standard errors can lead to substantial improvements in coverage rates even for samples with fifty or more clusters.We recommend that researchers routinely calculate the Bell-McCaffrey degrees-of-freedom adjustment to assess potential problems with conventional robust standard errors.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In contrast, if the working model proposed by Imbens and Kolesar (2016) is instead used (while still using OLS), then the the CR2 adjustments might differ depending on whether LSDV or the fixed effects estimator is used.”
From Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models (2018) · cites this paper as Imbens and Kolesar (2016)
In the citing paper
“However, a growing body of recent work has drawn attention to the shortcomings of CRVE methods when the data include only a small or moderate number of independent clusters (Cameron, Gelbach, and Miller 2008; Cameron and Miller 2015; Imbens and Kolesar 2016; MacKinnon and Webb 2016).”
From Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models (2018) · cites this paper as Imbens and Kolesar 2016
In the citing paper
“In particular, Wald tests based on CRVE can have rejection rates far in excess of the nominal Type I error.”
From Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models (2018) · cites this paper as Imbens and Kolesar 2016
In the citing paper
“This problem is compounded by the fact that the performance of standard Wald tests depends on features of the study design beyond just the total number of clusters, which can make it difficult to determine whether standard, asymptotically valid CRVE methods are accurate.”
From Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models (2018) · cites this paper as Imbens and Kolesar 2016
Citations
Cites
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Inference with Difference-in-Differences and Other Panel Data
Donald, S. G., & Lang, K. (2007) · Review of Economics and Statistics · doi:10.1162/rest.89.2.221 · Cited by 1,324
Heteroskedasticity-robust Standard Errors for Fixed Effects Panel Data Regression
Stock, J. H., & Watson, M. W. (2008) · Econometrica · doi:10.1111/j.0012-9682.2008.00821.x · Cited by 609
Research works printed but not linked: 19
- Behrens Walter U, 1929, Landwirtschaftliche Jahrbücher, 68, 807
- Bell Robert M, 2002, Survey Methodology, 28, 169
- 10.2307/1911269 · DOI · Paper page
- 10.1016/j.jeconom.2008.08.003 · DOI · Paper page
- 10.1111/j.1469-1809.1939.tb02205.x · DOI
- Hausman Jerry A, 2011, NBER working paper, 17698
- 10.1080/01621459.1975.10479877 · DOI · Paper page
- 10.1198/jbes.2009.08046 · DOI · Paper page
- Imbens Guido W, 2012, NBER working paper, 18478
- 10.1093/biomet/73.1.13 · DOI · Paper page
- 10.1214/aos/1176351062 · DOI · Paper page
- 10.1111/0008-4085.00147 · DOI · Paper page
- 10.1016/0304-4076(85)90158-7 · DOI · Paper page
- 10.1214/aos/1176349025 · DOI · Paper page
- 10.1016/0304-4076(86)90021-7 · DOI · Paper page
- Scheffé Henry, 1970, Journal of the American Statistical Association, 65, 1501
- 10.1080/01621459.1971.10482315 · DOI
- 10.1093/biomet/38.3-4.330 · DOI · Paper page
- 10.2307/1912934 · DOI · Paper page
Cited by
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841
Designing Difference in Difference Studies: Best Practices for Public Health Policy Research
Wing, C., Simon, K., & Bello-Gomez, R. A. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-013507 · Cited by 1,470
The Econometrics of Randomized Experiments
(2017) · Handbook of Economic Field Experiments · doi:10.1016/bs.hefe.2016.10.003 · Cited by 562
Pustejovsky, J. E., & Tipton, E. (2018) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2016.1247004 · Cited by 492
Econometric Methods for Program Evaluation
Abadie, A., & Cattaneo, M. D. (2018) · Annual Review of Economics · doi:10.1146/annurev-economics-080217-053402 · Cited by 431
Cluster-robust inference: A guide to empirical practice
MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2022.04.001 · Cited by 407
Cross-Country Trends in Affective Polarization
Boxell, L., Gentzkow, M., & Shapiro, J. (2020) · doi:10.3386/w26669 · Cited by 359
An Introductory Guide to Event Study Models
Miller, D. L. (2023) · Journal of Economic Perspectives · doi:10.1257/jep.37.2.203 · Cited by 360
The wild bootstrap for few (treated) clusters
MacKinnon, J. G., & Webb, M. D. (2018) · The Econometrics Journal · doi:10.1111/ectj.12107 · Cited by 317
Inference in Regression Discontinuity Designs with a Discrete Running Variable
Kolesár, M., & Rothe, C. (2018) · American Economic Review · doi:10.1257/aer.20160945 · Cited by 269
Self-citationBias and consistency in three-way gravity models
Weidner, M., & Zylkin, T. (2021) · Journal of International Economics · doi:10.1016/j.jinteco.2021.103513 · Cited by 201
Inference in Differences-in-Differences with Few Treated Groups and Heteroskedasticity
Ferman, B., & Pinto, C. (2019) · The Review of Economics and Statistics · doi:10.1162/rest_a_00759 · Cited by 178
Stock Market Wealth and the Real Economy: A Local Labor Market Approach
Chodorow-Reich, G., Nenov, P. T., & Simsek, A. (2021) · American Economic Review · doi:10.1257/aer.20200208 · Cited by 165
Affective Polarization in Comparative and Longitudinal Perspective
Garzia, D., Ferreira da Silva, F., & Maye, S. (2023) · Public Opinion Quarterly · doi:10.1093/poq/nfad004 · Cited by 137
Inference with few heterogeneous clusters
Ibragimov, R., & Müller, U. K. (2016) · Review of Economics and Statistics · doi:10.1162/rest_a_00545 · Cited by 99
Asymptotic Theory for Clustered Samples
Hansen, B. E., & Lee, S. (2019) · Journal of Econometrics · doi:10.1016/j.jeconom.2019.02.001 · Cited by 81
Rokicki, S., Cohen, J., Fink, G., Salomon, J. A., & Landrum, M. B. (2018) · Medical Care · doi:10.1097/mlr.0000000000000830 · Cited by 53
Wild Bootstrap Randomization Inference for Few Treated Clusters
MacKinnon, J. G., & Webb, M. D. (2019) · The Econometrics of Complex Survey Data · doi:10.1108/s0731-905320190000039003 · Cited by 21
Metadata
- Authors
- Imbens, G. W., & Kolesár, M.
- Venue
- Review of Economics and Statistics
- Publisher
- MIT Press
- Published
- 2016-10
- Volume
- 98
- Issue
- 4
- Pages
- 701-712
- ISSN
- 0034-6535
- 1530-9142
- Cited by
- 236
- References
- 5
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.