The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Robust Standard Errors in Small Samples: Some Practical Advice
Abstract
We study the properties of heteroskedasticity-robust confidence intervals for regression parameters. We show that confidence intervals based on a degrees-of-freedom correction suggested by Bell and McCaffrey (2002) are a natural extension of a principled approach to the Behrens-Fisher problem. We suggest a further improvement for the case with clustering. We show that these standard errors can lead to substantial improvements in coverage rates even for samples with fifty or more clusters.We recommend that researchers routinely calculate the Bell-McCaffrey degrees-of-freedom adjustment to assess potential problems with conventional robust standard errors.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In contrast, if the working model proposed by Imbens and Kolesar (2016) is instead used (while still using OLS), then the the CR2 adjustments might differ depending on whether LSDV or the fixed effects estimator is used.”
From Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models (2018) · cites this paper as Imbens and Kolesar (2016)
In the citing paper
“However, a growing body of recent work has drawn attention to the shortcomings of CRVE methods when the data include only a small or moderate number of independent clusters (Cameron, Gelbach, and Miller 2008; Cameron and Miller 2015; Imbens and Kolesar 2016; MacKinnon and Webb 2016).”
From Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models (2018) · cites this paper as Imbens and Kolesar 2016
In the citing paper
“In particular, Wald tests based on CRVE can have rejection rates far in excess of the nominal Type I error.”
From Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models (2018) · cites this paper as Imbens and Kolesar 2016
In the citing paper
“This problem is compounded by the fact that the performance of standard Wald tests depends on features of the study design beyond just the total number of clusters, which can make it difficult to determine whether standard, asymptotically valid CRVE methods are accurate.”
From Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models (2018) · cites this paper as Imbens and Kolesar 2016
Citations
Cites
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Inference with Difference-in-Differences and Other Panel Data
Donald, S. G., & Lang, K. (2007) · Review of Economics and Statistics · doi:10.1162/rest.89.2.221 · Cited by 1,324
Heteroskedasticity-robust Standard Errors for Fixed Effects Panel Data Regression
Stock, J. H., & Watson, M. W. (2008) · Econometrica · doi:10.1111/j.0012-9682.2008.00821.x · Cited by 609
Research works printed but not linked: 19
- Behrens Walter U, 1929, Landwirtschaftliche Jahrbücher, 68, 807
- Bell Robert M, 2002, Survey Methodology, 28, 169
- 10.2307/1911269 · DOI · Paper page
- 10.1016/j.jeconom.2008.08.003 · DOI · Paper page
- 10.1111/j.1469-1809.1939.tb02205.x · DOI
- Hausman Jerry A, 2011, NBER working paper, 17698
- 10.1080/01621459.1975.10479877 · DOI · Paper page
- 10.1198/jbes.2009.08046 · DOI · Paper page
- Imbens Guido W, 2012, NBER working paper, 18478
- 10.1093/biomet/73.1.13 · DOI · Paper page
- 10.1214/aos/1176351062 · DOI · Paper page
- 10.1111/0008-4085.00147 · DOI · Paper page
- 10.1016/0304-4076(85)90158-7 · DOI · Paper page
- 10.1214/aos/1176349025 · DOI · Paper page
- 10.1016/0304-4076(86)90021-7 · DOI · Paper page
- Scheffé Henry, 1970, Journal of the American Statistical Association, 65, 1501
- 10.1080/01621459.1971.10482315 · DOI
- 10.1093/biomet/38.3-4.330 · DOI · Paper page
- 10.2307/1912934 · DOI · Paper page
Cited by
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841
Designing Difference in Difference Studies: Best Practices for Public Health Policy Research
Wing, C., Simon, K., & Bello-Gomez, R. A. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-013507 · Cited by 1,470
Pustejovsky, J. E., & Tipton, E. (2018) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2016.1247004 · Cited by 492
Cluster-robust inference: A guide to empirical practice
MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2022.04.001 · Cited by 407
Cross-Country Trends in Affective Polarization
Boxell, L., Gentzkow, M., & Shapiro, J. (2020) · doi:10.3386/w26669 · Cited by 359
Metadata
- Authors
- Imbens, G. W., & Kolesár, M.
- Venue
- Review of Economics and Statistics
- Publisher
- MIT Press
- Published
- 2016-10
- Volume
- 98
- Issue
- 4
- Pages
- 701-712
- ISSN
- 0034-6535
- 1530-9142
- Cited by
- 236
- References
- 5
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Robust Standard Errors in Small Samples: Some Practical Advice. https://app.cochat.ai/papers/10.1162/rest_a_00552
- The paper
- Imbens, G. W., & Kolesár, M. (2016). Robust Standard Errors in Small Samples: Some Practical Advice. Review of Economics and Statistics. https://doi.org/10.1162/rest_a_00552
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
