Let's put garbage-can regressions and garbage-can probits where they belong
Abstract
Many social scientists believe that dumping long lists of explanatory variables into linear regression, probit, logit, and other statistical equations will successfully “control” for the effects of auxiliary factors. Encouraged by convenient software and ever more powerful computing, researchers also believe that this conventional approach gives the true explanatory variables the best chance to emerge. The present paper argues that these beliefs are false, and that without intensive data analysis, linear regression models are likely to be inaccurate. Instead, a quite different and less mechanical research methodology is needed, one that integrates contemporary powerful statistical methods with deep substantive knowledge and classic data—analytic techniques of creative engagement with the data.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Longitudinal designs are prone to residual confounding and other forms of bias that limit causal inference.
“Longitudinal designs provide stronger evidence, but are prone to residual confounding [[36], [37], [38]] and other forms of bias that limit causal inference [[39], [40], [41], [42], [43]].”
From Improving sleep quality leads to better mental health: A meta-analysis of randomised controlled trials (2021) · cites this paper as [36]
Cites
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,834 · see citing works
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,028 · see citing works
Draper, N. R., & Smith, H. (2005) · Technometrics · doi:10.1198/tech.2005.s303 · Cited by 17,993 · see citing works
The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies
Ragin, C. C. (2014) · doi:10.1525/9780520957350 · Cited by 2,130 · see citing works
Beyond the Running Tally: Partisan Bias in Political Perceptions
Bartels, L. M. (2002) · Political Behavior · doi:10.1023/a:1021226224601 · Cited by 1,468 · see citing works
Cleveland, W. S. (1985) · Medical Entomology and Zoology · Cited by 1,367 · see citing works
Graphs in Statistical Analysis
Anscombe, F. J. (1973) · The American Statistician · doi:10.2307/2682899 · Cited by 1,070 · see citing works
Graphs in Statistical Analysis
Anscombe, F. J. (1973) · The American Statistician · doi:10.1080/00031305.1973.10478966 · Cited by 976 · see citing works
The Phantom Menace: Omitted Variable Bias in Econometric Research
Clarke, K. A. (2005) · Conflict Management and Peace Science · doi:10.1080/07388940500339183 · Cited by 484 · cited 1× · see citing works
“(For rigorous and enlightening treatment of this point, see Clarke, 2005.)”
BackgroundAn Estimator for Some Binary-Outcome Selection Models Without Exclusion Restrictions
Sartori, A. E. (2003) · Political Analysis · doi:10.1093/pan/mpg001 · Cited by 469 · see citing works
T OWARD A N EW P OLITICAL M ETHODOLOGY : Microfoundations and ART
Achen, C. H. (2002) · Annual Review of Political Science · doi:10.1146/annurev.polisci.5.112801.080943 · Cited by 457 · see citing works
Self-citationStrategic Interaction and the Statistical Analysis of International Conflict
Signorino, C. S. (1999) · American Political Science Review · doi:10.2307/2585396 · Cited by 419 · see citing works
An Introduction to Regression Graphics
Goldman, A. S., Cook, R. D., & Weisberg, S. (1995) · Technometrics · doi:10.2307/1269921 · Cited by 361 · see citing works
An Introduction to Regression Graphics
Cook, R. D., & Weisberg, S. (1994) · Wiley series in probability and statistics · doi:10.1002/9780470316863 · Cited by 331 · see citing works
Political Science: The Science of Politics
Weisberg, H. F. (2008) · Cited by 281 · see citing works
International Relations Theory, Foreign Policy Substitutability, and “Nice” Laws
Most, B. A., & Starr, H. E. (1984) · World Politics · doi:10.2307/2010380 · Cited by 220 · see citing works
Causal Complexity and the Study of Politics
Braumoeller, B. F. (2003) · Political Analysis · doi:10.1093/pan/mpg012 · Cited by 179 · see citing works
Coleman, D. E., & Cleveland, W. (1987) · Technometrics · doi:10.1080/00401706.1987.10488193 · Cited by 132 · see citing works
Strategic Misspecification in Regression Models
Signorino, C. S., & Yılmaz, K. (2003) · American Journal of Political Science · doi:10.1111/1540-5907.00039 · Cited by 131 · see citing works
Rule of Three, Let It Be? When More Really Is Better
Oneal, J. R., & Russett, B. M. (2005) · Conflict Management and Peace Science · doi:10.1080/07388940500339209 · Cited by 108 · see citing works
Constructing Multivariate Analyses (of Dangerous Dyads)
Ray, J. L. (2005) · Conflict Management and Peace Science · doi:10.1080/07388940500339175 · Cited by 89 · see citing works
The Elements of Graphing Data.
UPTON, G. J. G., & Cleveland, W. S. (1988) · Biometrics · doi:10.2307/2531926 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Scott, A. J., Webb, T. L., Martyn-St James, M., Rowse, G., & Weich, S. (2021) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2021.101556 · Cited by 1,094 · see citing works
Thinking Clearly About Correlations and Causation: Graphical Causal Models for Observational Data
Rohrer, J. M. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245917745629 · Cited by 1,002 · see citing works
“Models resulting from such an approach have been labeled “garbage-can regressions” (Achen, 2005) because the idea that the inclusion of a multitude of control variables will necessarily improve (and will not worsen) causal inference is a methodological urban legend at best (Spector & Brannick,…”
BackgroundA Tale of Two Cultures: Contrasting Quantitative and Qualitative Research
Mahoney, J., & Goertz, G. (2006) · Political Analysis · doi:10.1093/pan/mpj017 · Cited by 904 · see citing works
The Phantom Menace: Omitted Variable Bias in Econometric Research
Clarke, K. A. (2005) · Conflict Management and Peace Science · doi:10.1080/07388940500339183 · Cited by 484 · see citing works
Multi-method evidence for when and how climate-related disasters contribute to armed conflict risk
Ide, T., Brzoska, M., Donges, J. F., & Schleussner, C.-F. (2020) · Global Environmental Change · doi:10.1016/j.gloenvcha.2020.102063 · Cited by 168 · see citing works
On malaria and the duration of civil war
Bagozzi, B. E. (2016) · Journal of Conflict Resolution · doi:10.1177/0022002714550202 · Cited by 38 · see citing works
- Authors
- Achen, C. H.
- Affiliations
- Princeton University (US)
- Corresponding author
- Achen, Christopher H.
- Venue
- Conflict Management and Peace Science
- Publisher
- SAGE Publications
- Published
- 2005-09
- Volume
- 22
- Issue
- 4
- Pages
- 327-339
- ISSN
- 0738-8942 (ISSN-L)
- 1549-9219
- OpenAlex
- W2164062141
- Cited by
- 576
- References
- 22
- Citations by year
- 2026: 28 · 2025: 16 · 2024: 54 · 2023: 29 · 2022: 36 · 2021: 50 · 2020: 62 · 2019: 27 · 2018: 29 · 2017: 25 · 2016: 34 · 2015: 35 · 2014: 26 · 2013: 31 · 2012: 27
- FWCI
- 32.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Conflict Management and Peace Science (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Qualitative Comparative Analysis Research
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- probit model, model specification, predictive factors, auxiliary variables, model misspecification, data analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
