The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
A General Framework for Comparing Predictions and Marginal Effects across Models
Abstract
Many research questions involve comparing predictions or effects across multiple models. For example, it may be of interest whether an independent variable’s effect changes after adding variables to a model. Or, it could be important to compare a variable’s effect on different outcomes or across different types of models. When doing this, marginal effects are a useful method for quantifying effects because they are in the natural metric of the dependent variable and they avoid identification problems when comparing regression coefficients across logit and probit models. Despite advances that make it possible to compute marginal effects for almost any model, there is no general method for comparing these effects across models. In this article, the authors provide a general framework for comparing predictions and marginal effects across models using seemingly unrelated estimation to combine estimates from multiple models, which allows tests of the equality of predictions and effects across models. The authors illustrate their method to compare nested models, to compare effects on different dependent or independent variables, to compare results from different samples or groups within one sample, and to assess results from different types of models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preacher, K. J., & Hayes, A. F. (2008) · Behavior Research Methods · doi:10.3758/brm.40.3.879 · Cited by 34,117
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Specification Tests in Econometrics
Hausman, J. A. (1978) · Econometrica · doi:10.2307/1913827 · Cited by 18,820
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550
A comparison of methods to test mediation and other intervening variable effects
MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.83 · Cited by 9,406 · cited 5×
“For an overview, see MacKinnon et al. (2002) and Preacher and Hayes (2008).”
BackgroundInfluentialMicroeconometrics: Methods and Applications
Cameron, A. C., & Trivedi, P. K. (2005) · doi:10.1017/cbo9780511811241 · Cited by 8,092
Journal of Economic Literature
(2019) · AEA Papers and Proceedings · doi:10.1257/pandp.109.633 · Cited by 7,468
Tests of Equality Between Sets of Coefficients in Two Linear Regressions
Chow, G. C. (1960) · Econometrica · doi:10.2307/1910133 · Cited by 6,488
Regression Models for Categorical Dependent Variables Using Stata
Guzmán, J. (2013) · Puerto Rico health sciences journal · Cited by 5,304
Regression Models for Categorical and Limited Dependent Variables
Calvin, J. A. (1998) · Technometrics · doi:10.1080/00401706.1998.10485496 · Cited by 5,201
Discrete Multivariate Analysis: Theory and Practice.
Beniger, J. R., Bishop, Y. M. M., Feinberg, S. E., Holland, P. W., Light, R. J., & Mosteller, F. (1975) · Contemporary Sociology · doi:10.2307/2063625 · Cited by 5,108
Maximum Likelihood Estimation of Misspecified Models
White, H. (1982) · Econometrica · doi:10.2307/1912526 · Cited by 4,098
Making the most of statistical analyses: improving interpretation and presentation
King, G., Tomz, M., & Wittenberg, J. (2000) · American Journal of Political Science · doi:10.2307/2669316 · Cited by 3,232 · cited 1×
“…using the delta method (Agresti 2013:72–77; Bishop, Fienberg, and Holland 1975:486– 97), bootstrapping (Efron and Tibshirani 1993), or simulation (King, Tomz, and Wittenberg 2000).2 The Wald tests in equations (3) and (4) can also be used to test predictions and marginal effects across models.”
MethodPreacher, K. J., & Kelley, K. (2011) · Psychological Methods · doi:10.1037/a0022658 · Cited by 3,060
USING THE CORRECT STATISTICAL TEST FOR THE EQUALITY OF REGRESSION COEFFICIENTS
PATERNOSTER, R., BRAME, R., MAZEROLLE, P., & PIQUERO, A. (1998) · Criminology · doi:10.1111/j.1745-9125.1998.tb01268.x · Cited by 2,742
A statistical model for the analysis of ordinal level dependent variables
McKelvey, R. D., & Zavoina, W. (1975) · The Journal of Mathematical Sociology · doi:10.1080/0022250x.1975.9989847 · Cited by 2,599
Statistical Methods for Comparing Regression Coefficients Between Models
Clogg, C. C., Petkova, E., & Haritou, A. (1995) · American Journal of Sociology · doi:10.1086/230638 · Cited by 2,072
Qualitative Response Models: A Survey
Amemiya, T. (1994) · Studies in Econometric Theory · doi:10.4337/9781035362707.00034 · Cited by 1,971
Assessing proportionality in the proportional odds model for ordinal logistic regression.
Brant, R. (1990) · Biometrics · doi:10.2307/2532457 · Cited by 1,864
The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant
Gelman, A., & Stern, H. (2006) · The American Statistician · doi:10.1198/000313006x152649 · Cited by 1,407 · cited 2×
“…differences by examining only the statistical significance of an effect within each model; this approach has been criticized because it is necessary to directly test cross-model differences to determine their statistical significance (Gelman and Stern 2006; Mustillo, Lizardo, and McVeigh 2018).”
BackgroundMethodInfluentialKarlson, K. B., Holm, A., & Breen, R. (2012) · Sociological Methodology · doi:10.1177/0081175012444861 · Cited by 1,204
PRACTITIONERS’ CORNER: Computing Robust Standard Errors for Within‐groups Estimators *
Arellano, M. (1987) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.1987.mp49004006.x · Cited by 1,191
Comparing Logit and Probit Coefficients Across Groups
ALLISON, P. D. (1999) · Sociological Methods & Research · doi:10.1177/0049124199028002003 · Cited by 1,047
Asymptotic Theory for Econometricians
(1984) · Elsevier eBooks · doi:10.1016/c2009-0-21869-1 · Cited by 540
Interpreting and Understanding Logits, Probits, and Other Nonlinear Probability Models
Breen, R., Karlson, K. B., & Holm, A. (2018) · Annual Review of Sociology · doi:10.1146/annurev-soc-073117-041429 · Cited by 468 · cited 1×
“Marginal effects do not have this limitation and are thus appropriate for comparing effects across logit and probit models (Breen, Karlson, and Holm 2018).”
MethodUsing Heterogeneous Choice Models to Compare Logit and Probit Coefficients Across Groups
Williams, R. (2009) · Sociological Methods & Research · doi:10.1177/0049124109335735 · Cited by 436
Using Predictions and Marginal Effects to Compare Groups in Regression Models for Binary Outcomes
Long, J. S., & Mustillo, S. A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118799374 · Cited by 385
Self-citationComputing Adjusted Risk Ratios and Risk Differences in Stata
Norton, E. C., Miller, M. M., & Kleinman, L. C. (2013) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x1301300304 · Cited by 300
Editors’ Comment: A Few Guidelines for Quantitative Submissions
Mustillo, S. A., Lizardo, O. A., & McVeigh, R. M. (2018) · American Sociological Review · doi:10.1177/0003122418806282 · Cited by 190
On the Formulation of Wald Tests of Nonlinear Restrictions
Phillips, P. C. B., & Park, J. Y. (1988) · Econometrica · doi:10.2307/1911359 · Cited by 173
(2012) · SpringerReference · doi:10.1007/springerreference_205246 · Cited by 168
Computation of Standard Errors
Dowd, B. E., Greene, W. H., & Norton, E. C. (2013) · Health Services Research · doi:10.1111/1475-6773.12122 · Cited by 98
Regression Models for Nominal and Ordinal Outcomes
Long, J. S. (2014) · SAGE Publications Ltd eBooks · doi:10.4135/9781446288146.n9 · Cited by 59
Self-citationCorrelations and Nonlinear Probability Models
Breen, R., Holm, A., & Karlson, K. B. (2014) · Sociological Methods & Research · doi:10.1177/0049124114544224 · Cited by 49
What Social Psychology Can Contribute to the Study of Sex, Gender, and Sexual Orientation
Mize, T. D. (2015) · Sociology Compass · doi:10.1111/soc4.12331 · Cited by 21
Self-citation
Cited by
A gendered pandemic: Childcare, homeschooling, and parents' employment during COVID‐19
Petts, R. J., Carlson, D. L., & Pepin, J. R. (2021) · Gender, Work & Organization · doi:10.1111/gwao.12614 · Cited by 417
Changes in US Parents’ Domestic Labor During the Early Days of the COVID‐19 Pandemic
Carlson, D. L., Petts, R. J., & Pepin, J. R. (2022) · Sociological Inquiry · doi:10.1111/soin.12459 · Cited by 320
Bonikowski, B., Feinstein, Y., & Bock, S. (2021) · American Journal of Sociology · doi:10.1086/717103 · Cited by 88
Metadata
- Authors
- Mize, T. D., Doan, L., & Long, J. S.
- Venue
- Sociological Methodology
- Publisher
- SAGE Publications
- Published
- 2019-06-20
- Volume
- 49
- Issue
- 1
- Pages
- 152-189
- ISSN
- 0081-1750 (ISSN-L)
- 1467-9531
- OpenAlex
- W2953305535
- Cited by
- 364
- References
- 35
- FWCI
- 21.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.