CoChat

The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index; 1 of the 69 sources its claims cite were checked for support.

Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models

Open access
Pustejovsky, James E., Tipton, Elizabeth · 2018 · Journal of Business & Economic Statistics

Abstract

In panel data models and other regressions with unobserved effects, fixed effects estimation is often paired with cluster-robust variance estimation (CRVE) to account for heteroscedasticity and un-modeled dependence among the errors. Although asymptotically consistent, CRVE can be biased downward when the number of clusters is small, leading to hypothesis tests with rejection rates that are too high. More accurate tests can be constructed using bias-reduced linearization (BRL), which corrects the CRVE based on a working model, in conjunction with a Satterthwaite approximation for t-tests. We propose a generalization of BRL that can be applied in models with arbitrary sets of fixed effects, where the original BRL method is undefined, and describe how to apply the method when the regression is estimated after absorbing the fixed effects. We also propose a small-sample test for multiple-parameter hypotheses, which generalizes the Satterthwaite approximation for t-tests. In simulations covering a wide range of scenarios, we find that the conventional cluster-robust Wald test can severely over-reject while the proposed small-sample test maintains Type I error close to nominal levels. The proposed methods are implemented in an R package called clubSandwich. This article has online supplementary materials.

Overview

CoChat Summary

TL;DR

The paper generalizes bias-reduced linearization (CR2) cluster-robust variance estimation to fixed effects models and proposes an approximate Hotelling’s T² (AHT) test for multi-parameter hypotheses. Simulations and two examples show the AHT test controls Type I error far better than standard CR1 Wald tests with \(m-1\) degrees of freedom, especially with few clusters or high-dimensional hypotheses.

Key findings

  • The proposed CR2 adjustment remains well-defined in models with arbitrary fixed effects by using a Moore–Penrose inverse, and the AHT test extends Satterthwaite-type small-sample corrections to multi-parameter hypotheses.
  • In simulations, the standard CR1 test with \(m-1\) df severely over-rejected: with 15 clusters, maximum Type I error ranged from 0.127 (\(q=1\)) to 0.687 (\(q=6\)); with 50 clusters, it still ranged from 0.071 (\(q=1\)) to 0.186 (\(q=6\)).
  • The AHT test kept rejection rates near nominal: with \(m=15\), rates did not exceed 0.012 for \(\alpha=.01\), 0.055 for \(\alpha=.05\), and 0.106 for \(\alpha=.10\); with at least 30 clusters, AHT rates were between 0.019 and 0.054 across conditions.
  • In the Achievement Awards example, the joint moderation test had standard \(p=.00010\) versus AHT \(p=.16057\); in the minimum legal drinking age example, the Hausman test had standard \(p=.06283\) versus AHT \(p=.11886\).

Methods

  • Develops a generalized BRL/CR2 variance estimator for weighted least squares with arbitrary fixed effects, proves unbiasedness under a working model, and shows absorption equivalence for OLS with an identity working model; proposes the AHT test by approximating the cluster-robust Wald statistic with Hotelling’s T² and estimated degrees of freedom.
  • Simulation study covered 1,944 unique conditions: randomized block, cluster-randomized, and difference-in-differences designs; balanced and unbalanced configurations; complete data or 15% missing; \(m=15,30,50\) clusters; main text states \(n=6,18,30\) units per cluster (Appendix Table A1 lists 12,18,30); 50,000 datasets per condition; tests of dimension \(q=1,2,3,6\) at \(\alpha=.05\).
  • Empirical examples use the Achievement Awards randomized trial (35 female-student schools) and a state-by-year minimum legal drinking age panel for 1970–1983, comparing AHT tests to standard CR1 tests with \(m-1\) degrees of freedom.

Limitations

  • Stated: AHT tests can under-reject when the number of clusters is very small and the design is unbalanced, severely so for multi-dimensional constraints (e.g., \(q=3\), \(m=15\), unbalanced difference-in-differences); CR2 is costly or infeasible when within-cluster sample sizes are large (e.g., \(n_i>10,000\)); extensions to multiway clustering, GEE, 2SLS, and GMM remain future work.
  • Stated: Theorem 2 was revised to correct an error; the previous more general claim of WLS equivalence does not hold.
  • Evident from the text: the main simulation description says \(n=6,18,30\) units per cluster, while Appendix Table A1 lists \(n=12,18,30\).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1080/07350015.2016.1247004

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

1 Valid · 4 Not valid · 45 Not checked · 83 Own results and interpretations

81 more claims

Create a free account to see all 133 claims and to verify the 2 references still open.

See all claims — free account
  • It may state two separate claims as one. 31
  • Probably not a substantial claim. 20

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 46 references

Research works printed but not linked: 3

  1. Bell R. M., 2002, Survey Methodology, 28, 169
  2. Carter A. V., 2013, Technical Report, University of California, Santa Barbara, CA. Available at
  3. McCaffrey D. F., 2001, Proceedings of the Annual Meeting of the American Statistical Association

Cited by

See all 529

Metadata

Authors
Pustejovsky, J. E., & Tipton, E.
Affiliations
  • University of Wisconsin–Madison (US)
  • Columbia University (US)
Venue
Journal of Business & Economic Statistics
Published
2017-06-12
Volume
36
Issue
4
Pages
672-683
ISSN
  • 0735-0015 (ISSN-L)
  • 1537-2707
OpenAlex
W2229226246
Cited by
492
References
46
Citations by year
2026: 203 · 2025: 76 · 2024: 48 · 2023: 44 · 2022: 37 · 2021: 47 · 2020: 15 · 2019: 14 · 2018: 4 · 2017: 3
FWCI
14.90
Type
journal article
Language
en
Open access
Locations
  • Journal of Business and Economic Statistics (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
Keywords
cluster-robust variance estimation, fixed effects model, panel data, small sample inference, Satterthwaite approximation, Wald test, heteroscedasticity, over-rejection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-11-13, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-13 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models. https://app.cochat.ai/papers/10.1080/07350015.2016.1247004
The paper
Pustejovsky, J. E., & Tipton, E. (2018). Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models. Journal of Business & Economic Statistics. https://doi.org/10.1080/07350015.2016.1247004

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.