The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Does Regression Produce Representative Estimates of Causal Effects?
Abstract
With an unrepresentative sample, the estimate of a causal effect may fail to characterize how effects operate in the population of interest. What is less well understood is that conventional estimation practices for observational studies may produce the same problem even with a representative sample. Causal effects estimated via multiple regression differentially weight each unit's contribution. The “effective sample” that regression uses to generate the estimate may bear little resemblance to the population of interest, and the results may be nonrepresentative in a manner similar to what quasi‐experimental methods or experiments with convenience samples produce. There is no general external validity basis for preferring multiple regression on representative samples over quasi‐experimental or experimental methods. We show how to estimate the “multiple regression weights” that allow one to study the effective sample. We discuss alternative approaches that, under certain conditions, recover representative average causal effects. The requisite conditions cannot always be met.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
A Simple Test for Heteroscedasticity and Random Coefficient Variation
Breusch, T. S., & Pagan, A. R. (1979) · Econometrica · doi:10.2307/1911963 · Cited by 5,647
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
Empirical Strategies in Labor Economics
Angrist, J. D., & Krueger, A. B. (1999) · Handbook of Labor Economics · doi:10.1016/s1573-4463(99)03004-7 · Cited by 1,628
Nested Analysis as a Mixed-Method Strategy for Comparative Research
LIEBERMAN, E. S. (2005) · American Political Science Review · doi:10.1017/s0003055405051762 · Cited by 1,515
Randomization Analysis of Experimental Data: The Fisher Randomization Test Comment
Rubin, D. B. (1980) · Journal of the American Statistical Association · doi:10.2307/2287653 · Cited by 1,368
Counterfactuals and Causal Inference: Methods and Principles for Social Research
Morgan, S. L., & Winship, C. (2007) · doi:10.1017/cbo9780511804564 · Cited by 1,146
Jensen, N. M. (2003) · International Organization · doi:10.1017/s0020818303573040 · Cited by 1,017
Research works printed but not linked: 45
- 10.1111/j.1540-5907.2008.00354.x · DOI · Paper page
- 10.1515/jci-2012-0009 · DOI
- 10.1146/annurev.economics.050708.143235 · DOI · Paper page
- Bardhan Pranab, 2013, Little, Big: Two Ideas about Fighting Global Poverty, Boston Review
- 10.1037/0022-3514.51.6.1173 · DOI · Paper page
- 10.1162/00208180151140658 · DOI · Paper page
- 10.2307/271073 · DOI · Paper page
- 10.1037/h0040950 · DOI · Paper page
- Campbell Donald T., 1984, Can We Be Scientific in Applied Social Science?, Evaluation Studies Review Annual, 9, 26
- 10.1016/S1573-4412(84)02014-6 · DOI · Paper page
- 10.1017/S002081831000024X · DOI
- Davidson Russell, 2004, Econometric Theory and Methods
- 10.1257/jel.48.2.424 · DOI · Paper page
- 10.1177/1065912907306470 · DOI · Paper page
- Gerber Alan S., 2012, Field Experiments: Design and Analysis
- 10.1111/j.1540-5907.2009.00424.x · DOI · Paper page
- 10.1177/153244000100100305 · DOI
- Greene William H., 2008, Econometric Analysis
- 10.1016/j.jeconom.2009.09.006 · DOI · Paper page
- Hernan Miguel A., 2013, Causal Inference
- 10.1162/REST_a_00007 · DOI · Paper page
- 10.1093/pan/mpl013 · DOI · Paper page
- 10.1007/978-3-540-75892-1_6 · DOI
- 10.1111/j.1467-985X.2007.00527.x · DOI · Paper page
- 10.1198/016214504000001187 · DOI · Paper page
- 10.1093/biomet/87.3.706 · DOI · Paper page
- 10.1257/jel.48.2.399 · DOI · Paper page
- 10.1257/jel.47.1.5 · DOI · Paper page
- 10.1093/pan/mpj004 · DOI · Paper page
- 10.1093/biomet/73.1.13 · DOI · Paper page
- 10.1214/12-AOAS583 · DOI · Paper page
- Long J. Scott, 1997, Regression Models for Categorical and Limited Dependent Variables
- McCullagh Peter, 1999, Generalized Linear Models
- 10.1017/CBO9780511762888 · DOI · Paper page
- 10.1214/ss/1177012031 · DOI · Paper page
- 10.1007/978-1-4419-9782-1_10 · doi:10.1007/978-1-4419-9782-1_10
- 10.1093/pan/mpp011 · DOI
- 10.2307/2981697 · DOI · Paper page
- 10.1214/ss/1009212410 · DOI · Paper page
- 10.1214/aos/1176344064 · DOI · Paper page
- 10.1007/978-1-4612-4378-6 · DOI · Paper page
- 10.2307/1913012 · DOI · Paper page
- 10.1007/978-1-4419-9782-1 · DOI · Paper page
- 10.1097/EDE.0b013e318127181b · DOI · Paper page
- Wooldridge Jeffrey M., 2009, Introductory Econometrics: A Modern Approach
Cited by
Design-based analysis in Difference-In-Differences settings with staggered adoption
Athey, S., & Imbens, G. W. (2022) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.10.012 · Cited by 917
On the Use of Two-Way Fixed Effects Regression Models for Causal Inference with Panel Data
Imai, K., & Kim, I. S. (2021) · Political Analysis · doi:10.1017/pan.2020.33 · Cited by 623
“…Kim 2019) to the 2FE estimator in order to understand the causal interpretation of these widely used estimators within the nonparametric framework (see, e.g., Humphreys 2009; Aronow and Samii 2015; Solon, Haider, and Wooldridge 2015, for related work on causal inference with crosssectional data).”
MethodGeneralizability of heterogeneous treatment effect estimates across samples
Coppock, A., Leeper, T. J., & Mullinix, K. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1808083115 · Cited by 445
When Should We Use Unit Fixed Effects Regression Models for Causal Inference with Longitudinal Data?
Imai, K., & Kim, I. S. (2019) · American Journal of Political Science · doi:10.1111/ajps.12417 · Cited by 369
Information equivalence in survey experiments
Dafoe, A., Zhang, B., & Caughey, D. (2018) · Political Analysis · doi:10.1017/pan.2018.9 · Cited by 356
Metadata
- Authors
- Aronow, P. M., & Samii, C.
- Venue
- American Journal of Political Science
- Publisher
- Wiley
- Published
- 2015-04-28
- Volume
- 60
- Issue
- 1
- Pages
- 250-267
- ISSN
- 0092-5853 (ISSN-L)
- 1540-5907
- OpenAlex
- W1883588036
- Cited by
- 192
- References
- 10
- FWCI
- 7.59
- Type
- journal article
- Language
- en
- Update notices
- erratum (2017-01-31)
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Does Regression Produce Representative Estimates of Causal Effects? https://app.cochat.ai/papers/10.1111/ajps.12185
- The paper
- Aronow, P. M., & Samii, C. (2016). Does Regression Produce Representative Estimates of Causal Effects? American Journal of Political Science. https://doi.org/10.1111/ajps.12185
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
