Improved hypothesis testing for coefficients in generalized estimating equations with small samples of clusters
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
The sandwich standard error estimator is commonly used for making inferences about parameter estimates found as solutions to generalized estimating equations (GEE) for clustered data. The sandwich tends to underestimate the variability in the parameter estimates when the number of clusters is small, and reference distributions commonly used for hypothesis testing poorly approximate the distribution of Wald test statistics. Consequently, tests have greater than nominal type I error rates. We propose tests that use bias‐reduced linearization, BRL, to adjust the sandwich estimator and Satterthwaite or saddlepoint approximations for the reference distribution of resulting Wald t‐tests. We conducted a large simulation study of tests using a variety of estimators (traditional sandwich, BRL, Mancl and DeRouen's BC estimator, and a modification of an estimator proposed by Kott) and approximations to reference distributions under diverse settings that varied the distribution of the explanatory variables, the values of coefficients, and the degree of intra‐cluster correlation (ICC). Our new method generally worked well, providing accurate estimates of the variability of fitted coefficients and tests with near‐nominal type I error rates when the ICC is small. Our method works less well when the ICC is large, but it continues to out‐perform the traditional sandwich and other alternatives. Copyright © 2006 John Wiley & Sons, Ltd.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Third, extant methods for hypothesis testing based on BRL are limited to single-parameter constraints (Bell and McCaffrey 2002; McCaffrey and Bell 2006) and small-sample methods for multiple-parameter hypothesis tests remain lacking.”
From Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models (2018) · cites this paper as McCaffrey and Bell 2006
Citations
Cites
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Wells, K. B., Sherbourne, C., Schoenbaum, M., Duan, N., Meredith, L., Unützer, J., Miranda, J., Carney, M. F., & Rubenstein, L. V. (2000) · JAMA · doi:10.1001/jama.283.2.212 · Cited by 1,021
Research works printed but not linked: 20
- 10.1093/biomet/73.1.13 · DOI · Paper page
- Fellegi I, 1980, Approximate tests of independence and goodness of fit based on stratified multistage samples, Journal of the American Statistical Association, 75, 261, 10.1080/01621459.1980.10477462 · DOI
- 10.1080/03610929008830234 · DOI
- 10.1080/03610919408813167 · doi:10.1080/03610919408813167
- 10.1080/03610919508813280 · DOI · Paper page
- 10.1111/j.0006-341X.2001.00126.x · DOI · Paper page
- Bell RM, 2002, Bias reduction in standard errors for linear regression with multi‐stage samples, Survey Methodology, 28, 169
- 10.1002/sim.1142 · DOI · Paper page
- 10.1007/978-1-4899-3242-6 · DOI · Paper page
- Shah BV, 1997, SUDAAN User's Manual, Release 7.5
- 10.1093/biomet/77.3.485 · DOI · Paper page
- McCaffreyDF BellRM BottsCH.Generalizations of bias reduced linearization. Proceedings of the Annual Meeting of the American Statistical Association 5–9 August2001.
- 10.2307/1426607 · doi:10.2307/1426607
- 10.2307/2686101 · doi:10.2307/2686101
- 10.1198/16214502388618636 · doi:10.1198/16214502388618636
- KottPS.Linear regression in the face of specification error: model‐based exploration of randomization‐based techniques. Statistical Society of Canada Proceedings of the Survey Methods Section1996;39–47.
- 10.1093/biomet/88.1.287 · DOI
- Elliot M, 2003, Sampling with Field Burden Constraints: An Application to Sheltered Homeless and Low‐income Housed Women DRU‐3057‐NIDA
- 10.2307/2334280 · DOI
- 10.1007/978-1-4899-4541-9 · DOI · Paper page
Cited by
Pustejovsky, J. E., & Tipton, E. (2018) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2016.1247004 · Cited by 492
“The same authors also proposed small-sample corrections to single-parameter hypothesis tests using the BRL variance estimator, based on Satterthwaite (Bell and McCaffrey 2002) or saddlepoint approximations (McCaffrey and Bell 2006).”
InfluentialTipton, E., & Pustejovsky, J. E. (2015) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998615606099 · Cited by 362
“McCaffrey and Bell (2006) present evidence that, for tests of single regression coefficients, a saddlepoint approximation may provide even more accurate error control than the Satterthwaite approximation studied in Tipton (2015).”
BackgroundRokicki, S., Cohen, J., Fink, G., Salomon, J. A., & Landrum, M. B. (2018) · Medical Care · doi:10.1097/mlr.0000000000000830 · Cited by 53
Metadata
- Authors
- McCaffrey, D. F., & Bell, R. M.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2006-02-03
- Volume
- 25
- Issue
- 23
- Pages
- 4081-4098
- ISSN
- 0277-6715
- 1097-0258
- Cited by
- 29
- References
- 2
- Language
- en
- Licence
- onlinelibrary.wiley.com/termsAndConditions#vor (vor)
- doi.wiley.com/10.1002/tdm_license_1.1 (tdm)
- Funders
- National Science Foundation (00017630)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Improved hypothesis testing for coefficients in generalized estimating equations with small samples of clusters. https://app.cochat.ai/papers/10.1002/sim.2502
- The paper
- McCaffrey, D. F., & Bell, R. M. (2006). Improved hypothesis testing for coefficients in generalized estimating equations with small samples of clusters. Statistics in Medicine. https://doi.org/10.1002/sim.2502
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.