Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index; 1 of the 69 sources its claims cite were checked for support.
Abstract
In panel data models and other regressions with unobserved effects, fixed effects estimation is often paired with cluster-robust variance estimation (CRVE) to account for heteroscedasticity and un-modeled dependence among the errors. Although asymptotically consistent, CRVE can be biased downward when the number of clusters is small, leading to hypothesis tests with rejection rates that are too high. More accurate tests can be constructed using bias-reduced linearization (BRL), which corrects the CRVE based on a working model, in conjunction with a Satterthwaite approximation for t-tests. We propose a generalization of BRL that can be applied in models with arbitrary sets of fixed effects, where the original BRL method is undefined, and describe how to apply the method when the regression is estimated after absorbing the fixed effects. We also propose a small-sample test for multiple-parameter hypotheses, which generalizes the Satterthwaite approximation for t-tests. In simulations covering a wide range of scenarios, we find that the conventional cluster-robust Wald test can severely over-reject while the proposed small-sample test maintains Type I error close to nominal levels. The proposed methods are implemented in an R package called clubSandwich. This article has online supplementary materials.
Overview
CoChat Summary
TL;DR
The paper generalizes bias-reduced linearization (CR2) cluster-robust variance estimation to fixed effects models and proposes an approximate Hotelling’s T² (AHT) test for multi-parameter hypotheses. Simulations and two examples show the AHT test controls Type I error far better than standard CR1 Wald tests with \(m-1\) degrees of freedom, especially with few clusters or high-dimensional hypotheses.
Key findings
- The proposed CR2 adjustment remains well-defined in models with arbitrary fixed effects by using a Moore–Penrose inverse, and the AHT test extends Satterthwaite-type small-sample corrections to multi-parameter hypotheses.
- In simulations, the standard CR1 test with \(m-1\) df severely over-rejected: with 15 clusters, maximum Type I error ranged from 0.127 (\(q=1\)) to 0.687 (\(q=6\)); with 50 clusters, it still ranged from 0.071 (\(q=1\)) to 0.186 (\(q=6\)).
- The AHT test kept rejection rates near nominal: with \(m=15\), rates did not exceed 0.012 for \(\alpha=.01\), 0.055 for \(\alpha=.05\), and 0.106 for \(\alpha=.10\); with at least 30 clusters, AHT rates were between 0.019 and 0.054 across conditions.
- In the Achievement Awards example, the joint moderation test had standard \(p=.00010\) versus AHT \(p=.16057\); in the minimum legal drinking age example, the Hausman test had standard \(p=.06283\) versus AHT \(p=.11886\).
Methods
- Develops a generalized BRL/CR2 variance estimator for weighted least squares with arbitrary fixed effects, proves unbiasedness under a working model, and shows absorption equivalence for OLS with an identity working model; proposes the AHT test by approximating the cluster-robust Wald statistic with Hotelling’s T² and estimated degrees of freedom.
- Simulation study covered 1,944 unique conditions: randomized block, cluster-randomized, and difference-in-differences designs; balanced and unbalanced configurations; complete data or 15% missing; \(m=15,30,50\) clusters; main text states \(n=6,18,30\) units per cluster (Appendix Table A1 lists 12,18,30); 50,000 datasets per condition; tests of dimension \(q=1,2,3,6\) at \(\alpha=.05\).
- Empirical examples use the Achievement Awards randomized trial (35 female-student schools) and a state-by-year minimum legal drinking age panel for 1970–1983, comparing AHT tests to standard CR1 tests with \(m-1\) degrees of freedom.
Limitations
- Stated: AHT tests can under-reject when the number of clusters is very small and the design is unbalanced, severely so for multi-dimensional constraints (e.g., \(q=3\), \(m=15\), unbalanced difference-in-differences); CR2 is costly or infeasible when within-cluster sample sizes are large (e.g., \(n_i>10,000\)); extensions to multiway clustering, GEE, 2SLS, and GMM remain future work.
- Stated: Theorem 2 was revised to correct an error; the previous more general claim of WLS equivalence does not hold.
- Evident from the text: the main simulation description says \(n=6,18,30\) units per cluster, while Appendix Table A1 lists \(n=12,18,30\).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 4 Not valid · 45 Not checked · 83 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Evidence from a wide variety of contexts indicates that the asymptotic limiting distribution of robust Wald statistics can be a poor approximation when the number of clusters is small, even if corrections such as CR2 or CR3 are employed (Bell and McCaffrey 2002; Bertrand, Duflo, and Mullainathan 2004; Cameron, Gelbach, and Miller 2008).”
What the sources say
- Not matchedcites Bell and McCaffrey 2002
An author–year citation the paper’s reference list doesn’t settle. We don’t search these by name.
- Partly supportedcites Bertrand, Duflo, and Mullainathan 2004 matched to “How Much Should We Trust Differences-in-Differences Estimates?” (2004) sourceExperimental — accuracy not yet measured
Bertrand, Duflo and Mullainathan (2004) clearly support the core of the claim for their DD setting: the Introduction (p. 1) states that clustered-error solutions 'work well when the number of groups is large (e.g. 50 states) but fare poorly as the number of groups gets small'; §4.5 (p. 4.5) reports rejection rates of 9.5% with 20 states and 31.5% with only 6, §4.4 reports 15% with 6 states even under an arbitrary variance-covariance matrix, and footnote 31 (§4.6) notes 'at these small samples (as small as 12), the normal approximation to the t-distribution will clearly not work'. What our copy does NOT support is the qualifier 'even if corrections such as CR2 or CR3 are employed': no passage in our copy mentions CR2, CR3, or any small-sample degrees-of-freedom adjustment of that kind (the corrections examined are Moulton/White clustered SEs, AR(1) parametric, block bootstrap, aggregation, an arbitrary empirical variance-covariance matrix, and randomization inference). Likewise the framing 'a wide variety of contexts' overreaches this source, whose evidence is CPS-based DD minimum-wage simulations plus a survey of published DD papers, not a multi-context collection. (LLM, conf=0.78)
- Not checked — no full text of the sourcecites Cameron, Gelbach, and Miller 2008 matched to “Bootstrap-Based Improvements for Inference With Clustered Errors” (2008) source
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The approximate degrees of freedom of the AHT test can be much smaller than the number of clusters, particularly when the covariates involved in the test involve high leverage or are unbalanced across clusters.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Accounting for fixed effects also makes hypothesis test results invariant to how the regression is calculated (whether by LSDV or after absorption), which we see as a useful feature for ensuring the replicability of one's analysis.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unbalanced designs lead to rejection rates that are usually below the nominal α -just the opposite of how the standard test is affected by unbalance.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For both of these designs, unbalance leads to sharply reduced degrees of freedom.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
81 more claims
Create a free account to see all 133 claims and to verify the 2 references still open.
- It may state two separate claims as one. 31
- Probably not a substantial claim. 20
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“CR2 standard errors are designed for use with small-sample fixed effect models, and represent a conservative choice relative to conventional clustered standard errors.[46]”
From Life After Benchmark Saturation: A Case Study of CORE-Bench (2026) · cites this paper as [46]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To account for dependencies between effect sizes stemming from the same study and the small number of studies, we report estimates and standard errors based on cluster-robust covariance matrices (Pustejovsky & Tipton, 2018).”
From The Impact of COVID‐19‐Related School Closures on Student Achievement—A Meta‐Analysis (2022) · cites this paper as Pustejovsky & Tipton, 2018
The cited source says what the claim attributes to it.
In the citing paper
“we accounted for the hierarchical structure using robust inference”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Pustejovsky and Tipton, 2017
In the citing paper
“For many of the schemes only one to five studies are available, which does not allow for appropriate clustering of the effect sizes85,86.”
From Systematic review and meta-analysis of ex-post evaluations on the effectiveness of carbon pricing (2024) · cites this paper as [86]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ 4.1 Design · p. 17
“with m = 15, 30, or 50 cluster and n = 6, 18, or 30 units per cluster”
§ Appendix C Details of simulation study · p. 40
“| Units per cluster ( n ) | 12, 18, 30 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 1.3 Standard CRVE · p. 7
“where e i = Y i -X i ˆ β is the vector of residuals from cluster i”
§ 1.1 Econometric framework · p. 5
“Let U i = [ R i S i ] denote the set of predictors that are identified across multiple clusters, X i = [ R i S i T i ] denote the full set of predictors, α = ( β ′ , γ ′ , µ ′ ) ′ , and p = r + s + t .”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 5.1 Achievement Awards demonstration · p. 25
“H 0 : β 21 = β 22 = β 23 , with q = 3”
§ 5.1 Achievement Awards demonstration · p. 26
“| Moderation - upper half (q = 2) | Standard | 3.186 | 34.00 | 0.05393 |”
Citations
Cites
(2001) · RePEc: Research Papers in Economics · Cited by 27,577
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550 · cited 2×
“A common approach to inference in these settings is to use a cluster-robust variance estimator (CRVE; White 1984; Liang and Zeger 1986; Arellano 1987).”
How Much Should We Trust Differences-in-Differences Estimates?
Bertrand, M., Duflo, E., & Mullainathan, S. (2004) · The Quarterly Journal of Economics · doi:10.1162/003355304772839588 · Cited by 10,641 · cited 3×
“The need to cluster the standard errors by state, even when including state fixed effects, was highlighted by Bertrand, Duflo, and Mullainathan (2004), who showed that to do otherwise can lead to inappropriately small standard errors and hypothesis tests with incorrect rejection rates.”
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467 · cited 3×
“In the past decade, use of CRVEs has become standard practice for micro-economic researchers, as evidenced by coverage in major textbooks and review articles (e.g., Angrist and Pischke 2009; Wooldridge 2010; Cameron and Miller 2015).”
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
A Practitioner’s Guide to Cluster-Robust Inference
Colin Cameron, A., & Miller, D. L. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.317 · Cited by 5,037 · cited 10×
“Following Cameron and Miller (2015), we refer to this estimator as CR0.”
InfluentialBootstrap-Based Improvements for Inference With Clustered Errors
Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2008) · Review of Economics and Statistics · doi:10.1162/rest.90.3.414 · Cited by 4,417 · cited 4×
“Cameron, Gelbach, and Miller (2008) investigated a range of bootstrapping procedures that provide improved Type I error control in small samples, finding that a cluster wild-bootstrap technique was particularly accurate in small samples.”
InfluentialAn Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373 · cited 1×
“ive t-test developed by Bell & McCarey (2002) involves using a t() references distribution, with degrees of freedom estimated by a Satterthwaite approximation. The Satterthwaite approximation (Satterthwaite 1946) entails using degrees of freedom that are a function of the the rst two moments of the sampling distri…”
Robust Inference With Multiway Clustering
Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2011) · Journal of Business & Economic Statistics · doi:10.1198/jbes.2010.07136 · Cited by 3,203 · cited 1×
“2 estimator and AHT testing procedure for weighted least squares estimation of linear regression models. Extensions to linear regression models with clustering in multiple, non-nested dimensions (cf. Cameron et al. 2011) appear to be possible, and their utility should be further investigated. McCarey & Bell (2006)…”
MacKinnon, J. G., & White, H. (1985) · Journal of Econometrics · doi:10.1016/0304-4076(85)90158-7 · Cited by 1,879 · cited 2×
“We refer to this correction as CR2 because it extends the HC2 variance estimator for regressions with uncorrelated errors, which is exactly unbiased when the errors are homoscedastic (MacKinnon and White 1985).”
Inference with Difference-in-Differences and Other Panel Data
Donald, S. G., & Lang, K. (2007) · Review of Economics and Statistics · doi:10.1162/rest.89.2.221 · Cited by 1,324
Asymptotic Theory for Econometricians.
Robinson, P. M., & White, H. (1988) · Economica · doi:10.2307/2554258 · Cited by 1,305
PRACTITIONERS’ CORNER: Computing Robust Standard Errors for Within‐groups Estimators *
Arellano, M. (1987) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.1987.mp49004006.x · Cited by 1,191 · cited 2×
“A common approach to inference in these settings is to use a cluster-robust variance estimator (CRVE; White 1984; Liang and Zeger 1986; Arellano 1987).”
Cluster-Sample Methods in Applied Econometrics
Wooldridge, J. M. (2003) · American Economic Review · doi:10.1257/000282803321946930 · Cited by 867 · cited 1×
“The consistency property of CRVEs is asymptotic in the number of independent clusters (Wooldridge 2003).”
Regression discontinuity inference with specification error
Lee, D. S., & Card, D. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.003 · Cited by 853
Small sample adjustments for robust variance estimation with meta-regression.
Tipton, E. (2015) · Psychological Methods · doi:10.1037/met0000011 · Cited by 810 · cited 15×
“These results follow closely those from Tipton and Pustejovsky (2015), which also found that even with extreme model misspecification the Type I error of the CR2S test was close to nominal.”
InfluentialSelf-citationA Covariance Estimator for GEE with Improved Small‐Sample Properties
Mancl, L. A., & DeRouen, T. A. (2001) · Biometrics · doi:10.1111/j.0006-341x.2001.00126.x · Cited by 656 · cited 1×
“This estimator, denoted CR3, closely approximates the jackknife resampling estimator (Mancl and DeRouen 2001; Bell andMcCaffrey 2002).”
Inference with “Difference in Differences” with a Small Number of Policy Changes
Conley, T. G., & Taber, C. R. (2011) · Review of Economics and Statistics · doi:10.1162/rest_a_00049 · Cited by 633 · cited 1×
“Conley and Taber (2011) proposed methods for hypothesis testing in a difference-in-differences setting where the number of treated units is small and fixed, while the number of untreated units increases asymptotically.”
Wild Bootstrap Inference for Wildly Different Cluster Sizes
MacKinnon, J. G., & Webb, M. D. (2017) · Journal of Applied Econometrics · doi:10.1002/jae.2508 · Cited by 454
Mastering 'metrics: the path from cause to effect
(2015) · Choice Reviews Online · doi:10.5860/choice.189854 · Cited by 381
Tipton, E., & Pustejovsky, J. E. (2015) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998615606099 · Cited by 362 · cited 8×
“These findings follow closely those from Tipton and Pustejovsky (2015), which indicated that the Type I error of the AHT test was close to nominal even when the working model was quite discrepant from the data-generating model.”
InfluentialSelf-citationBias reduction in standard errors for linear regression with multi-stage samples
McCaffrey, D. F., & Bell, R. M. (2003) · Quality Engineering · Cited by 318
t-Statistic Based Correlation and Heterogeneity Robust Inference
Ibragimov, R., & Müller, U. K. (2010) · Journal of Business & Economic Statistics · doi:10.1198/jbes.2009.08046 · Cited by 299
Asymptotic properties of a robust variance matrix estimator for panel data when is large
Hansen, C. B. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2006.10.009 · Cited by 293 · cited 3×
“Hansen (2007) provided one justification for the use of this reference distribution by identifying conditions under which Z converges in distribution to t(m−1) as the within-cluster sample sizes grow large, with m fixed (see also Donald & Lang 2007). Ibragimov & Müller (2010) proposed a weighting technique derived so…”
The Effects of High Stakes High School Achievement Awards: Evidence from a Randomized Trial
Angrist, J., & Lavy, V. (2009) · American Economic Review · doi:10.1257/aer.99.4.1384 · Cited by 270 · cited 3×
“In an analysis of a longitudinal cluster-randomized trial with 35 clusters, Angrist and Lavy (2009) observed that the BRL correction makes a difference for inferences.”
Hrong-Tai Fai, A., & Cornelius, P. L. (1996) · Journal of Statistical Computation and Simulation · doi:10.1080/00949659608811740 · Cited by 260
On the testing of correlated effects with panel data
Arellano (1993) · Journal of Econometrics · doi:10.1016/0304-4076(93)90040-c · Cited by 247
Robust Standard Errors in Small Samples: Some Practical Advice
Imbens, G. W., & Kolesár, M. (2016) · Review of Economics and Statistics · doi:10.1162/rest_a_00552 · Cited by 236 · cited 9×
“Consequently, Tipton (2015) and Imbens and Kolesar (2016) argued that t-tests based on CRVE should routinely use the CR2 variance estimator and the Satterthwaite degrees of freedom, even when m appears to be large.”
InfluentialEstimating multi-way error components models with unbalanced data structures
Davis, P. (2002) · Journal of Econometrics · doi:10.1016/s0304-4076(01)00087-2 · Cited by 234 · cited 2×
“For further details on sequential absorption, see Davis (2002).”
Linear Algebra and Matrix Analysis for Statistics
Banerjee, S., & Roy, A. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/b17040 · Cited by 218
Bootstrap-Based Improvements for Inference with Clustered Errors
Cameron, A. C., Gelbach, J., & Miller, D. (2007) · National Bureau of Economic Research · doi:10.3386/t0344 · Cited by 182
Mastering ‘Metrics: The Path from Cause to Effect
Gardebroek, C. (2015) · European Review of Agricultural Economics · doi:10.1093/erae/jbv011 · Cited by 180
The Minimum Legal Drinking Age and Public Health
Carpenter, C., & Dobkin, C. (2011) · Journal of Economic Perspectives · doi:10.1257/jep.25.2.133 · Cited by 161 · cited 1×
“As a second illustration, we draw on a panel data analysis described in Angrist and Pischke (2014; see also Carpenter and Dobkin 2011).”
Pan, W., & Wall, M. M. (2002) · Statistics in Medicine · doi:10.1002/sim.1142 · Cited by 150 · cited 3×
“…(2015) compared the performance of the AHT test to several other possible approximate F-tests, including adaptations of the tests introduced by Pan and Wall (2002) and Zhang (2012), as well as adaptations of tests based on eigen-decompositions proposed by Fai and Cornelius (1996) and Cai and…”
InfluentialAsymptotic Behavior of a t-Test Robust to Cluster Heterogeneity
Carter, A. V., Schnepel, K. T., & Steigerwald, D. G. (2017) · The Review of Economics and Statistics · doi:10.1162/rest_a_00639 · Cited by 120 · cited 4×
“The estimated degrees of freedom are indicative of the precision of the standard errors, and thus provide diagnostic information that is similar to the effective sample size measure proposed by Carter et al. (2013). We therefore recommend that the degrees of freedom be reported along with standard errors and p-values…”
InfluentialThirty Years of Heteroskedasticity-Robust Inference
MacKinnon, J. G. (2013) · Recent Advances and Future Directions in Causality, Prediction, and Specification Analysis · doi:10.1007/978-1-4614-1653-1_17 · Cited by 114 · cited 5×
“downward and associated hypothesis tests can have Type-I error rates considerably in excess of nominal levels when based on samples with only a small or moderate number of clusters (e.g., Webb & MacKinnon 2013). Cameron & Miller (2015) provide a thorough review of this literature, including a discussion of cur…”
InfluentialA solution to the multivariate behrens-fisher problem
Nel, D. G., & Van Der Merwe, C. A. (1986) · Communications in Statistics - Theory and Methods · doi:10.1080/03610928608829342 · Cited by 109 · cited 3×
“The method is a generalization of the Satterthwaite correction proposed by Bell & McCaffrey (2002) for single parameter constraints. Our work is related to a stream of recent literature that has examined methods for cluster-robust inference with a small number of clusters. Conley & Taber (2011) proposed methods for hy…”
InfluentialModified Nel and Van der Merwe test for the multivariate Behrens–Fisher problem
Krishnamoorthy, K., & Yu, J. (2003) · Statistics & Probability Letters · doi:10.1016/j.spl.2003.10.012 · Cited by 106 · cited 1×
“Nel and van der Merwe (1986) proposed an AHT-type test for equality of multivariate means across two samples with unequal variance-covariance matrices (i.e., the multivariate Behrens–Fisher problem; see also Krishnamoorthy and Yu 2004).”
Inference with few heterogeneous clusters
Ibragimov, R., & Müller, U. K. (2016) · Review of Economics and Statistics · doi:10.1162/rest_a_00545 · Cited by 99 · cited 1×
“Ibragimov and Müller (2010; 2016) proposed cluster-robust t-tests that maintain the nominal Type I error rate by reweighting withincluster estimators of the target parameter.”
Improving the brown-forsythe solution to the generalized behrens-fisher problem
Mehrotra, D. V. (1997) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610919708813431 · Cited by 73 · cited 1×
“…need to focus on other approximations to the reference distribution itself, such as an F distribution in which both the numerator and denominator degrees of freedom are estimated (see Mehrotra 1997) or computational approximations such as the cluster-wild bootstrap (MacKinnon and Webb in press).”
White, H., White, H., & Swanson, N. R. (2012) · Springer eBooks · doi:10.1007/978-1-4614-1653-1 · Cited by 73 · cited 1×
“…still tends to under-estimate the true variance of β̂ because the degree of bias depends not only on the number of clusters m, but also on skewness of the covariates and unbalance across clusters (Carter, Schnepel, and Steigerwald 2013; MacKinnon 2013; Cameron and Miller 2015; Young 2016).”
A New Test of Linear Hypotheses in OLS Regression Under Heteroscedasticity of Unknown Form
Cai, L., & Hayes, A. F. (2007) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998607302628 · Cited by 53 · cited 1×
“…compared the performance of the AHT test to several other possible approximate F-tests, including adaptations of the tests introduced by Pan and Wall (2002) and Zhang (2012), as well as adaptations of tests based on eigen-decompositions proposed by Fai and Cornelius (1996) and Cai and Hayes (2008).”
White, H. (1984) · Asymptotic Theory for Econometricians · doi:10.1016/b978-0-12-746650-7.50012-8 · Cited by 35 · cited 1×
“A common approach to inference in these settings is to use a cluster-robust variance estimator (CRVE; White 1984; Liang and Zeger 1986; Arellano 1987).”
An approximate degrees of freedom test for heteroscedastic two-way ANOVA
Zhang, J. (2011) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2011.07.023 · Cited by 34
McCaffrey, D. F., & Bell, R. M. (2006) · Statistics in Medicine · doi:10.1002/sim.2502 · Cited by 29 · cited 3×
“The same authors also proposed small-sample corrections to single-parameter hypothesis tests using the BRL variance estimator, based on Satterthwaite (Bell and McCaffrey 2002) or saddlepoint approximations (McCaffrey and Bell 2006).”
Influential« L’Actualité Économique, Revue d’analyse économique »
de rédaction, L. C. (1983) · L Actualité économique · doi:10.7202/601039ar · Cited by 13
(2014) · Psychological Methods · doi:10.1037/met0000011.supp · Cited by 3
Research works printed but not linked: 1
- McCaffrey, Daniel F, Robert M Bell, and Carsten H Botts. 2001. 'Generalizations of biased reduced linearization.' In Proceedings of the Annual Meeting of the American Statistical Association . 1994.
Cited by
Designing Difference in Difference Studies: Best Practices for Public Health Policy Research
Wing, C., Simon, K., & Bello-Gomez, R. A. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-013507 · Cited by 1,470
Cluster-robust inference: A guide to empirical practice
MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2022.04.001 · Cited by 407
“Young (2016) proposes a related method that uses CV1 instead of CV2.”
MethodInvestigating the replicability of preclinical cancer biology
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.71601 · Cited by 462
“, when effect sizes could be determined for both original and replication effects), we accounted for the hierarchical structure using robust inference (Bell and McCaffrey, 2002; Pustejovsky and Tipton, 2017) to account for the nesting structure as described in ‘Assessing candidate moderators of replication success met…”
MethodDigital interventions for the treatment of depression: a meta-analytic review
Moshe, I., Terhorst, Y., Philippi, P., Domhardt, M., Cuijpers, P., Cristea, I., Pulkki-Råback, L., Baumeister, H., & Sander, L. B. (2021) · Psychological Bulletin · doi:10.1037/bul0000334 · Cited by 390
clubSandwich: Cluster-Robust (Sandwich) Variance Estimators with Small-Sample Corrections
Pustejovsky, J. E. (2016) · CRAN: Contributed Packages · doi:10.32614/cran.package.clubsandwich · Cited by 216
Self-citation
Metadata
- Authors
- Pustejovsky, J. E., & Tipton, E.
- Affiliations
- University of Wisconsin–Madison (US)
- Columbia University (US)
- Venue
- Journal of Business & Economic Statistics
- Publisher
- Informa UK Limited
- Published
- 2017-06-12
- Volume
- 36
- Issue
- 4
- Pages
- 672-683
- ISSN
- 0735-0015 (ISSN-L)
- 1537-2707
- OpenAlex
- W2229226246
- Cited by
- 492
- References
- 46
- Citations by year
- 2026: 203 · 2025: 76 · 2024: 48 · 2023: 44 · 2022: 37 · 2021: 47 · 2020: 15 · 2019: 14 · 2018: 4 · 2017: 3
- FWCI
- 14.90
- Type
- journal article
- Language
- en
- Update notices
- correction (2023-02-22)
- Open access
- green
- arxiv.org
- Locations
- Journal of Business and Economic Statistics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Keywords
- cluster-robust variance estimation, fixed effects model, panel data, small sample inference, Satterthwaite approximation, Wald test, heteroscedasticity, over-rejection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-13, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-13 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.