Marginal mean weighting through stratification: A generalized method for evaluating multivalued and multiple treatments with nonexperimental data
The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
Propensity score matching and stratification enable researchers to make statistical adjustment for a large number of observed covariates in nonexperimental data. These methods have recently become popular in psychological research. Yet their applications to evaluations of multi-valued and multiple treatments are limited. The inverse-probability-of-treatment weighting method, though suitable for evaluating multi-valued and multiple treatments, often generates results that are not robust when only a portion of the population provides support for causal inference or when the functional form of the propensity score model is misspecified. The marginal mean weighting through stratification (MMW-S) method promises a viable nonparametric solution to these problems. By computing weights on the basis of stratified propensity scores, MMW-S adjustment equates the pretreatment composition of multiple treatment groups under the assumption that unmeasured covariates do not confound the treatment effects given the observed covariates. Analyzing data from a weighted sample, researchers can estimate a causal effect by computing the difference between the estimated average potential outcomes associated with alternative treatments within the analysis of variance framework. After providing an intuitive illustration of the theoretical rationale underlying the weighting method for causal inferences, the article demonstrates how to apply the MMW-S method to evaluations of treatments measured on a binary, ordinal, or nominal scale approximating a completely randomized experiment; to studies of multiple concurrent treatments approximating factorial randomized designs; and to moderated treatment effects approximating randomized block designs. The analytic procedure is illustrated with an evaluation of educational services for English language learners attending kindergarten in the United States.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In particular, we use the marginal mean weighting through stratification (MMWS) approach (Hong, 2012).”
From The effects of uppercase vs. lowercase letters on consumers’ perceptions and brand attitudes (2021) · cites this paper as Hong, 2012
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,265
An Introduction to Categorical Data Analysis
Agresti, A. (2007) · Wiley Series in Probability and Statistics · doi:10.1002/0470114754 · Cited by 6,689
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293
Austin, P. C. (2011) · Pharmaceutical Statistics · doi:10.1002/pst.433 · Cited by 3,949
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
A Generalization of Sampling Without Replacement From a Finite Universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.2307/2280784 · Cited by 3,011
Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review
Imbens, G. W. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023651 · Cited by 2,994
Estimating Causal Effects from Large Data Sets Using Propensity Scores
Rubin, D. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-8_part_2-199710151-00064 · Cited by 2,877
Bayesian inference for causal effects: The role of randomization
Rubin, D. B. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344064 · Cited by 2,503
Variable Selection for Propensity Score Models
Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., & Stürmer, T. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj149 · Cited by 2,336
The role of the propensity score in estimating dose-response functions
Imbens, G. (2000) · Biometrika · doi:10.1093/biomet/87.3.706 · Cited by 1,776
Aptitudes and Instructional Methods: A Handbook for Research on Interactions
McLeod, D. B., Cronbach, L. J., & Snow, R. E. (1978) · Journal for Research in Mathematics Education · doi:10.2307/748778 · Cited by 1,479
Children's social and scholastic lives in kindergarten: related spheres of influence?
Ladd, G. W., Birch, S. H., & Buhs, E. S. (1999) · Child Development · doi:10.1111/1467-8624.00101 · Cited by 1,136
Invited Commentary: Propensity Scores
Joffe, M. M., & Rosenbaum, P. R. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a010011 · Cited by 1,048
Rosenbaum, P. R. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478441 · Cited by 997
The Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies
Cochran, W. G. (1968) · Biometrics · doi:10.2307/2528036 · Cited by 961
Causal Inference With General Treatment Regimes: Generalizing the Propensity Score
Imai, K., & van Dyk, D. A. (2004) · Journal of the American Statistical Association · doi:10.1198/016214504000001187 · Cited by 868
Marginal Structural Models versus Structural nested Models as Tools for Causal inference
Robins, J. M. (2000) · The IMA Volumes in Mathematics and its Applications · doi:10.1007/978-1-4612-1284-3_2 · Cited by 550
Statistical Models in Epidemiology, the Environment, and Clinical Trials
(2000) · The IMA Volumes in Mathematics and its Applications · doi:10.1007/978-1-4612-1284-3 · Cited by 543
Average causal effects from nonrandomized studies: A practical guide and simulated example.
Schafer, J. L., & Kang, J. (2008) · Psychological Methods · doi:10.1037/a0014268 · Cited by 529
Combining Propensity Score Matching with Additional Adjustments for Prognostic Covariates
Thomas, N. (2006) · Matched Sampling for Causal Effects · doi:10.1017/cbo9780511810725.025 · Cited by 508
Analysis of Covariance: Its Nature and Uses
Cochran, W. G. (1957) · Biometrics · doi:10.2307/2527916 · Cited by 504
Statistics and Causal Inference: Comment: Which Ifs Have Causal Answers
Rubin, D. B. (1986) · Journal of the American Statistical Association · doi:10.2307/2289065 · Cited by 501
The Hispanic-White Achievement Gap in Math and Reading in the Elementary Grades
Reardon, S. F., & Galindo, C. (2009) · American Educational Research Journal · doi:10.3102/0002831209333184 · Cited by 476
An Introduction to Categorical Data Analysis
Bilder, C. R., & Tebbs, J. M. (2008) · Journal of the American Statistical Association · doi:10.1198/jasa.2008.s251 · Cited by 455
The prognostic analogue of the propensity score
Hansen, B. B. (2008) · Biometrika · doi:10.1093/biomet/asn004 · Cited by 371
Effects of Misspecification of the Propensity Score on Estimators of Treatment Effect
Drake, C. (1993) · Biometrics · doi:10.2307/2532266 · Cited by 347
Estimating and Using Propensity Scores with Partially Missing Data
D'Agostino, R. B., & Rubin, D. B. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474263 · Cited by 300
Models for Nonresponse in Sample Surveys
Little, R. J. A. (1982) · Journal of the American Statistical Association · doi:10.1080/01621459.1982.10477792 · Cited by 278
Tiihonen, J., Lönnqvist, J., Wahlbeck, K., Klaukka, T., Tanskanen, A., & Haukka, J. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.12.1358 · Cited by 277
Aptitude-treatment interaction as a framework for research on individual differences in learning.
Snow, R. E. (1989) · Cited by 274
Learning and individual differences: Advances in theory and research.
Ackerman, P. L., Glaser, R. A., & Sternberg, R. J. (1989) · Medical Entomology and Zoology · Cited by 242
Matching With Doses in an Observational Study of a Media Campaign Against Drug Abuse
Lu, B., Zanutto, E., Hornik, R., & Rosenbaum, P. R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753381896 · Cited by 236
Maternal Employment and Child Development: A Fresh Look Using Newer Methods.
Hill, J. L., Waldfogel, J., Brooks-Gunn, J., & Han, W.-J. (2005) · Developmental Psychology · doi:10.1037/0012-1649.41.6.833 · Cited by 224
Journal of the Royal Statistical Society. Series A (General)
Jstor (1987) · Cited by 184
Marginal Mean Weighting Through Stratification: Adjustment for Selection Bias in Multilevel Data
Hong, G. (2010) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998609359785 · Cited by 149
Self-citationZanutto, E., Lu, B., & Hornik, R. (2005) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986030001059 · Cited by 108
Votruba‐Drzal, E., Li-Grining, C. P., & Maldonado‐Carreño, C. (2008) · Child Development · doi:10.1111/j.1467-8624.2008.01170.x · Cited by 102
Huang, I., Frangakis, C., Dominici, F., Diette, G. B., & Wu, A. W. (2005) · Health Services Research · doi:10.1111/j.1475-6773.2005.00352.x · Cited by 89
Causal Inference for Time-Varying Instructional Treatments
Hong, G., & Raudenbush, S. W. (2008) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998607307355 · Cited by 74
Self-citationRobinson, J. P. (2008) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373708317742 · Cited by 59
Hong, G., & Hong, Y. (2009) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373708328259 · Cited by 55
Self-citationThe General Linear Model and Direct Standardization: A Comparison
Little, R. J. A., & Pullum, T. W. (1979) · Sociological Methods & Research · doi:10.1177/004912417900700407 · Cited by 35
Citrome, L., Reist, C., Palmer, L. A., Montejano, L. B., Lenhart, G. M., Cuffel, B. J., Harnett, J., & Sanders, K. N. (2009) · Schizophrenia Research · doi:10.1016/j.schres.2009.09.023 · Cited by 21
Zhu, B., Kulkarni, P. M., Stensland, M. D., & Ascher‐Svanum, H. (2007) · Current Medical Research and Opinion · doi:10.1185/030079907x233322 · Cited by 16
Cited by
Statistical mediation analysis with a multicategorical independent variable
Hayes, A. F., & Preacher, K. J. (2014) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12028 · Cited by 3,403
“…certain alternative explanations, and more complex statistical approaches than we discuss here can be used in non-experimental studieswhen causal inference is less justified due to limitations of the design (such as non-random assignment; see, for example, Hong, 2012; Muth en, 2011; Pearl, 2012).”
MethodJiang, F., & McComas, W. F. (2015) · International Journal of Science Education · doi:10.1080/09500693.2014.1000426 · Cited by 167
“causal inference, the propensity score analysis was performed within the marginal mean weighting through stratification (MMW-S) approach (Hong, 2012).”
BackgroundMethodInfluentialEstimating causal effects for multivalued treatments: a comparison of approaches
Linden, A., Uysal, S. D., Ryan, A. M., & Adams, J. (2015) · Statistics in Medicine · doi:10.1002/sim.6768 · Cited by 165
WeightIt: Weighting for Covariate Balance in Observational Studies
Greifer, N. (2017) · doi:10.32614/cran.package.weightit · Cited by 102
Sheth, D. S., Cafri, G., Paxton, E. W., & Namba, R. S. (2016) · The Journal of Arthroplasty · doi:10.1016/j.arth.2016.03.018 · Cited by 87
“All models we fit address device/ person-level confounding using average treatment effect propensity score weights [25,26].”
BackgroundMethod
Metadata
- Authors
- Hong, G.
- Affiliations
- University of Chicago (US)
- Corresponding author
- Hong, Guanglei
- ORCID
- Hong, Guanglei
- Venue
- Psychological Methods
- Published
- 2012-03
- Volume
- 17
- Issue
- 1
- Pages
- 44-60
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W1993260957
- Cited by
- 105
- References
- 46
- Citations by year
- 2026: 5 · 2025: 3 · 2024: 7 · 2023: 12 · 2022: 4 · 2021: 6 · 2020: 9 · 2019: 5 · 2018: 13 · 2017: 5 · 2016: 9 · 2015: 19 · 2014: 4 · 2013: 2 · 2012: 1
- FWCI
- 2.25
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Spencer Foundation
- Social Sciences and Humanities Research Council of Canada
- National Academy of Education
- William T. Grant Foundation
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- multiple treatments, nonparametric causal inference, inverse probability of treatment weighting, factorial design, kindergarten, non-experimental data
- MeSH terms
- Analysis of Variance, Child, Data Interpretation, Statistical, Humans, Learning, Longitudinal Studies, Models, Theoretical, Research Design, Vocabulary, Models, Statistical, Program Evaluation, Selection Bias, Causality, Randomized Controlled Trials as Topic, Statistics, Nonparametric, Propensity Score
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.