The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation
Open accessAbstract
A basic feature of many field experiments is that investigators are only able to randomize clusters of individuals—such as households, communities, firms, medical practices, schools or classrooms—even when the individual is the unit of interest. To recoup the resulting efficiency loss, some studies pair similar clusters and randomize treatment within pairs. However, many other studies avoid pairing, in part because of claims in the literature, echoed by clinical trials standards organizations, that this matched-pair, cluster-randomization design has serious problems. We argue that all such claims are unfounded. We also prove that the estimator recommended for this design in the literature is unbiased only in situations when matching is unnecessary; its standard error is also invalid. To overcome this problem without modeling assumptions, we develop a simple design-based estimator with much improved statistical properties. We also propose a model-based approach that includes some of the benefits of our design-based estimator as well as the estimator in the literature. Our methods also address individual-level noncompliance, which is common in applications but not allowed for in most existing methods. We show that from the perspective of bias, efficiency, power, robustness or research costs, and in large or small samples, pairing should be used in cluster-randomized experiments whenever feasible; failing to do so is equivalent to discarding a considerable fraction of one’s data. We develop these techniques in the context of a randomized evaluation we are conducting of the Mexican Universal Health Insurance Program.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
McLachlan, G., & Peel, D. (2000) · Wiley Series in Probability and Statistics · doi:10.1002/0471721182 · Cited by 10,991
Addelman, S. (1979) · Technometrics · doi:10.1080/00401706.1979.10489788 · Cited by 6,750
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230
Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference
Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007) · Political Analysis · doi:10.1093/pan/mpl013 · Cited by 4,205
Self-citationConsort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510
Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review
Imbens, G. W. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023651 · Cited by 2,994
CONSORT statement: extension to cluster randomised trials
Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.328.7441.702 · Cited by 2,557
On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9
Splawa-Neyman, J., Dabrowska, D. M., & Speed, T. P. (1990) · Statistical Science · doi:10.1214/ss/1177012031 · Cited by 1,627
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236
(1958) · Medical Entomology and Zoology · Cited by 1,059
Misunderstandings Between Experimentalists and Observationalists about Causal Inference
Imai, K., King, G., & Stuart, E. A. (2008) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2007.00527.x · Cited by 906
Self-citationSimple sample size calculation for cluster-randomized trials
Hayes, R. J., & Bennett, S. (1999) · International Journal of Epidemiology · doi:10.1093/ije/28.2.319 · Cited by 822
IMPACT OF VITAMIN A SUPPLEMENTATION ON CHILDHOOD MORTALITY
Sommer, A., Djunaedi, E., Loeden, A. A., Tarwotjo, I., West, K., Tilden, R., Mele, L., & The Aceh Study Group (1986) · The Lancet · doi:10.1016/s0140-6736(86)91157-8 · Cited by 680
Rubin, D. B. (1990) · Statistical Science · doi:10.1214/ss/1177012032 · Cited by 665
Statistical analysis and optimal design for cluster randomized trials.
Raudenbush, S. W. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.2.173 · Cited by 636
SYMPOSIUM ON CHD PREVENTION TRIALS: DESIGN ISSUES IN TESTING LIFE STYLE INTERVENTION
CORNFIELD, J. (1978) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112592 · Cited by 539
Sobel, M. E. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000636 · Cited by 468
Pitfalls of and controversies in cluster randomization trials
Donner, A., & Klar, N. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.416 · Cited by 442
(2000) · Technometrics · doi:10.1080/00401706.2000.10486038 · Cited by 412
Maldonado, G. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.2.422 · Cited by 399
Interference Between Units in Randomized Experiments
Rosenbaum, P. R. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001112 · Cited by 371
King, G., Gakidou, E., Imai, K., Lakin, J., Moore, R. T., Nall, C., Ravishankar, N., Vargas, M., Téllez-Rojo, M. M., Ávila, J. E. H., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60239-7 · Cited by 324
Self-citationCampbell, M. K., Mollison, J., & Grimshaw, J. M. (2001) · Statistics in Medicine · doi:10.1002/1097-0258(20010215)20:3<391::aid-sim800>3.0.co;2-z · Cited by 299
Covariate-based constrained randomization of group-randomized trials
Moulton, L. H. (2004) · Clinical Trials · doi:10.1191/1740774504cn024oa · Cited by 283
ON DESIGN CONSIDERATIONS AND RANDOMIZATION-BASED INFERENCE FOR COMMUNITY INTERVENTION TRIALS
GAIL, M. H., MARK, S. D., CARROLL, R. J., GREEN, S. B., & PEE, D. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960615)15:11<1069::aid-sim220>3.0.co;2-q · Cited by 248
Strategies for Improving Precision in Group-Randomized Experiments
Raudenbush, S. W., Martinez, A., & Spybrook, J. (2007) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373707299460 · Cited by 226
(2006) · Matched Sampling for Causal Effects · doi:10.1017/cbo9780511810725.033 · Cited by 211
Optimal multivariate matching before randomization
Greevy, R. (2004) · Biostatistics · doi:10.1093/biostatistics/5.2.263 · Cited by 205
Rubin, D. B. (1991) · Biometrics · doi:10.2307/2532381 · Cited by 202
Designing and analysing case-control studies to exploit independence of genotype and exposure
Umbach, D. M., & Weinberg, C. R. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970815)16:15<1753::aid-sim597>3.0.co;2-e · Cited by 179
Some Aspects of the Design and Analysis of Cluster Randomization Trials
Donner, A. (1998) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/1467-9876.00100 · Cited by 178
Evidence-based health policy: three generations of reform in Mexico
Frenk, J., Sepúlveda, J., Gómez-Dantés, O., & Knaul, F. (2003) · The Lancet · doi:10.1016/s0140-6736(03)14803-9 · Cited by 161
King, G., Gakidou, E. E., Ravishankar, N., Moore, R. T., Lakin, J. M., Vargas, M., Téllez‐Rojo, M. M., Hernández‐Ávila, J. E., Ávila, M. H., & Llamas, H. H. (2007) · Journal of Policy Analysis and Management · doi:10.1002/pam.20279 · Cited by 148
Self-citationAspects of statistical design for the community intervention trial for smoking cessation (COMMIT)
Gail, M. H., Byar, D. P., Pechacek, T. F., & Corle, D. K. (1992) · Controlled Clinical Trials · doi:10.1016/0197-2456(92)90026-v · Cited by 144
Selected Statistical Issues in Group Randomized Trials
Feng, Z., Diehr, P. H., Peterson, A. V., & McLerran, D. (2001) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.22.1.167 · Cited by 144
Design and Analysis of Group-Randomized Trials: A Review of Recent Practices
Varnell, S. P., Murray, D. M., Janega, J. B., & Blitstein, J. L. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.393 · Cited by 141
The Core Analytics of Randomized Experiments for Social Research
Bloom, H. S. (2012) · SAGE Publications Ltd eBooks · doi:10.4135/9781446212165.n9 · Cited by 132
Analysis of data arising from a stratified design with the cluster as unit of randomization
Donner, A. P., & Donald, A. W. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060106 · Cited by 121
Analysis of cluster randomized cross‐over trial data: a comparison of methods
Turner, R., White, I. R., & Croudace, T. J. (2006) · Statistics in Medicine · doi:10.1002/sim.2537 · Cited by 112
The effect of matching on the power of randomized community intervention studies
Martin, D. C., Diehr, P. K., Perrin, E. B., & Koepsell, T. D. (1993) · Statistics in Medicine · doi:10.1002/sim.4780120315 · Cited by 106
The Effect of High School Matriculation Awards: Evidence from Randomized Trials
Angrist, J. D., & Lavy, V. (2002) · National Bureau of Economic Research · doi:10.3386/w9389 · Cited by 106
Frangakis, C., Rubin, D. B., & Zhou, X.-H. (2002) · Biostatistics · doi:10.1093/biostatistics/3.2.147 · Cited by 97
Imai, K. (2008) · Statistics in Medicine · doi:10.1002/sim.3337 · Cited by 93
Self-citationConfidence interval construction for effect measures arising from cluster randomization trials
DONNER, A. P., & Klar, N. S. (1993) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(93)90050-b · Cited by 92
Small, D. S., Have, T. R. T., & Rosenbaum, P. R. (2008) · Journal of the American Statistical Association · doi:10.1198/016214507000000897 · Cited by 79
Using Cluster Randomized Field Experiments to Study Voting Behavior
Arceneaux, K. (2005) · The Annals of the American Academy of Political and Social Science · doi:10.1177/0002716205277804 · Cited by 75
Optimal Permutation Tests for the Analysis of Group Randomized Trials
Braun, T., & Feng, Z. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382336 · Cited by 72
Standardization: A Technique to Control for Extraneous Variables
Kalton, G. (1968) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2985676 · Cited by 70
Rubin, D. B., & Stuart, E. A. (2006) · The Annals of Statistics · doi:10.1214/009053606000000407 · Cited by 69
STATISTICAL METHODOLOGY FOR PAIRED CLUSTER DESIGNS1
Donner, A. P. (1987) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a114735 · Cited by 50
Extending CONSORT to include cluster trials
Campbell, M. J. (2004) · BMJ · doi:10.1136/bmj.328.7441.654 · Cited by 46
Misunderstandings among experimentalists and observationalists about causal inference
Imai, K., King, G., & Stuart, E. A. (2007) · Cited by 40
Self-citationEstimation of a common odds ratio in paired‐cluster randomization designs
Donner, A. P., & Hauck, W. W. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080509 · Cited by 18
Thompson, S. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980930)17:18<2149::aid-sim897>3.0.co;2-x · Cited by 12
Research works printed but not linked: 6
- Higgins, J. and Green, S., eds. (2006). Cochrane Handbook for Systematic Review of Interventions 4.2.5 [Updated September 2006]. Wiley, Chichester, UK.
- Hill, J. L., Rubin, D. B. and Thomas, N. (1999). The design of the New York school choice scholarship program evaluation. In Research Designs: Inspired by the Work of Donald Campbell (L. Bickman, ed.) 155–180. Sage, Thousand Oaks.
- Murray, D. M. (1998). Design and Analysis of Community Trials. Oxford Univ. Press, Oxford.
- Snedecor, G. W. and Cochran, W. G. (1989). Statistical Methods, 8th ed. Iowa State Univ. Press, Ames, IA.
- Klar, N. and Donner, A. (1998). Author’s reply. Statist. Med. 17 2151–2152. · doi:10.1002/(sici)1097-0258(19980930)17:18<2151::aid-sim967>3.0.co;2-j
- Wei, L. J. (1982). Interval estimation of location difference with incomplete data. Biometrika 69 249–251. · doi:10.1093/biomet/69.1.249
Cited by
Causal Inference without Balance Checking: Coarsened Exact Matching
Iacus, S. M., King, G., & Porro, G. (2012) · Political Analysis · doi:10.1093/pan/mpr013 · Cited by 3,563
Self-citationWhy Propensity Scores Should Not Be Used for Matching
King, G., & Nielsen, R. (2019) · Political Analysis · doi:10.1017/pan.2019.11 · Cited by 1,706
Self-citationCem: Coarsened Exact Matching in Stata
Blackwell, M., Iacus, S., King, G., & Porro, G. (2009) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900402 · Cited by 1,689
Self-citationMultivariate Matching Methods That Are Monotonic Imbalance Bounding
Iacus, S. M., King, G., & Porro, G. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm09599 · Cited by 967
Self-citationIn Pursuit of Balance: Randomization in Practice in Development Field Experiments
Bruhn, M., & McKenzie, D. (2009) · American Economic Journal: Applied Economics · doi:10.1257/app.1.4.200 · Cited by 788
Metadata
- Authors
- Imai, K., King, G., & Nall, C.
- Venue
- Statistical Science
- Publisher
- Institute of Mathematical Statistics
- Published
- 2009-02-01
- Volume
- 24
- Issue
- 1
- Pages
- 29-53
- ISSN
- 0883-4237 (ISSN-L)
- OpenAlex
- W2150319295
- Cited by
- 291
- References
- 56
- FWCI
- 9.57
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.