CoChat

Counterfactuals and Hypothesis Testing in Political Science

Open access
Fearon, James D. · 1991 · World Politics

The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.

Abstract

Scholars in comparative politics and international relations routinely evaluate causal hypotheses by referring to counterfactual cases where a hypothesized causal factor is supposed to have been absent. The methodological status and the viability of this very common procedure are unclear and are worth examining. How does the strategy of counterfactual argument relate, if at all, to methods of hypothesis testing based on the comparison of actual cases, such as regression analysis or Mill's Method of Difference? Are counterfactual thought experiments a viable means of assessing hypotheses about national and international outcomes, or are they methodologically invalid in principle? The paper addresses the first question in some detail and begins discussion of the second. Examples from work on the causes of World War I, the nonoccurrence of World War III, social revolutions, the breakdown of democratic regimes in Latin America, and the origins of fascism and corporatism in Europe illustrate the use, problems and potential of counterfactual argument in small-N-oriented political science research.

Overview

CoChat Summary

TL;DR

Fearon argues that counterfactual conditionals play a necessary and fundamental, if often implicit, role in how political scientists assess causal hypotheses, and that analysts face a choice between two and only two strategies for empirically testing a causal claim: the "actual case" strategy and the "counterfactual case" strategy. The paper's primary concern is small-N research, where negative degrees of freedom make counterfactual argument logically necessary to establish the causal importance of any one variable.

Key findings

  • When experimental control and replication are not possible, analysts can either (1) imagine that C had been absent and ask whether E would have occurred, or (2) search for other actual cases resembling the case in question except that C is absent or has a different value. Both strategies aim to solve the same statistical problem: the analyst begins with one case and at least one explanatory variable, meaning negative degrees of freedom, and legitimate causal imputations cannot be made on that basis.
  • In large-N, non-experimental work, counterfactual assumptions enter via the "ceteris paribus" assumption (E(x'e)=0). Fearon argues this assumption is credible "if and only if" the counterfactual proposition (PI) is credible: "If the cases in the sample had assumed different values on the independent variables, the contents of the error term would not have differed systematically."
  • The strategies differ in how they provide confirmation: the actual case strategy relies on frequency or magnitude of association across actual cases, while the counterfactual strategy cannot meaningfully assess frequencies and instead relies on general principles, theories, or laws distinct from the hypothesis being tested, plus historical facts relevant to the counterfactual scenario.
  • Assessing relative causal weight in the counterfactual strategy requires a proliferation of counterfactual cases — to argue A was a more important cause than B, at least two scenarios are needed (A present/B absent versus B present/A absent).

Methods

  • Conceptual and methodological argument developed in three sections: distinguishing the "actual case" and "counterfactual" strategies of hypothesis testing; illustrating the counterfactual strategy in practice through examples from international relations and comparative politics; and returning to theoretical issues about the link between causal arguments and counterfactual propositions.
  • Illustrative examples include the "cult of the offensive" hypothesis about World War I (Van Evera 1984, Snyder 1984), structural versus domestic explanations of Soviet foreign policy (following Waltz 1979, 1981), and works such as Moore (1966), Skocpol (1979), and Luebbert (1987).
  • No formal data collection or statistical analysis is reported; the paper is a methodological/theoretical essay.

Limitations

  • The paper states that arguments about what would have happened in multiple counterfactual scenarios "will be very imprecise and uncertain," and that in the counterfactual strategy "there is no formal criterion for gauging the risk of error associated with some independent variable — all depends instead on the plausibility of arguments about 'what would have happened.'"
  • The paper acknowledges that the counterfactual strategy for empirically checking a causal hypothesis "seems only indirectly empirical, since the confirmation it provides depends principally on other theories."
  • Evident from the text: the provided full text is truncated mid-sentence in the second section ("Second, I will consider the role of counterfactuals in sever…"), so the paper's examples in practice and its third section on theoretical issues are only partially available here.

Model-written summary of a shortened version of the paper’s full text (it is longer than we read). Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2307/2010470

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Image 1
From the PDF
Image 2
From the PDF
Image 3
From the PDF

Verification

Claims the paper makes

0 Valid · 1 Not valid · 0 Not checked · 20 Own results and interpretations

7 more claims

Create a free account to see all 21 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 3
  • It may state two separate claims as one. 6
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Citations

Cites

Show all 77 references

Research works printed but not linked: 8

  1. Ragin, 1987, The Comparative Method, 39
  2. George, 1985, Advances in Information Processing in Organizations, 2
  3. Gaddis John Lewis , “Nuclear Weapons and International Systemic Stability,” American Academy of Arts and Sciences Occasional Paper No. 2 (Cambridge: AAAS, 1990).
  4. Polsby, 1982, What If?: Essays in Social Science Fiction
  5. Moore, 1966, Social Origins of Dictatorship and Democracy, 414
  6. Fischer, 1970, Historians' Fallacies, 15
  7. Lewis, 1973, Counterfactuals, 65
  8. Barry, 1980, Supertax, Political Studies, 28, 139

Cited by

Items 501–600 of 725

Metadata

Authors
Fearon, J. D.
Affiliations
University of California, Berkeley (US)
Corresponding author
Fearon, James D.
Venue
World Politics
Publisher
Project MUSE
Published
1991-01
Volume
43
Issue
2
Pages
169-195
ISSN
  • 0043-8871 (ISSN-L)
  • 1086-3338
OpenAlex
W1972181371
Cited by
729
References
74
Citations by year
2026: 32 · 2025: 16 · 2024: 14 · 2023: 40 · 2022: 40 · 2021: 36 · 2020: 35 · 2019: 32 · 2018: 36 · 2017: 17 · 2016: 22 · 2015: 20 · 2014: 24 · 2013: 25 · 2012: 20
FWCI
28.70
Type
journal article
Language
en
Open access
Locations
  • World Politics (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion)
  • Virtual Defense Library (Ministerio de Defensa) (submittedVersion)
Topics
  • Social Sciences › Sociology and Political Science › Qualitative Comparative Analysis Research
  • Social Sciences › Political Science and International Relations › Political Science Research and Education
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
Keywords
hypothesis testing, causal inference, comparative case study, small-N research, origins of World War I, social revolution, Latin America, origins of fascism, corporatism, international relations theory

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.