Counterfactuals and Hypothesis Testing in Political Science
Open accessThe DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Abstract
Scholars in comparative politics and international relations routinely evaluate causal hypotheses by referring to counterfactual cases where a hypothesized causal factor is supposed to have been absent. The methodological status and the viability of this very common procedure are unclear and are worth examining. How does the strategy of counterfactual argument relate, if at all, to methods of hypothesis testing based on the comparison of actual cases, such as regression analysis or Mill's Method of Difference? Are counterfactual thought experiments a viable means of assessing hypotheses about national and international outcomes, or are they methodologically invalid in principle? The paper addresses the first question in some detail and begins discussion of the second. Examples from work on the causes of World War I, the nonoccurrence of World War III, social revolutions, the breakdown of democratic regimes in Latin America, and the origins of fascism and corporatism in Europe illustrate the use, problems and potential of counterfactual argument in small-N-oriented political science research.
Overview
CoChat Summary
TL;DR
Fearon argues that counterfactual conditionals play a necessary and fundamental, if often implicit, role in how political scientists assess causal hypotheses, and that analysts face a choice between two and only two strategies for empirically testing a causal claim: the "actual case" strategy and the "counterfactual case" strategy. The paper's primary concern is small-N research, where negative degrees of freedom make counterfactual argument logically necessary to establish the causal importance of any one variable.
Key findings
- When experimental control and replication are not possible, analysts can either (1) imagine that C had been absent and ask whether E would have occurred, or (2) search for other actual cases resembling the case in question except that C is absent or has a different value. Both strategies aim to solve the same statistical problem: the analyst begins with one case and at least one explanatory variable, meaning negative degrees of freedom, and legitimate causal imputations cannot be made on that basis.
- In large-N, non-experimental work, counterfactual assumptions enter via the "ceteris paribus" assumption (E(x'e)=0). Fearon argues this assumption is credible "if and only if" the counterfactual proposition (PI) is credible: "If the cases in the sample had assumed different values on the independent variables, the contents of the error term would not have differed systematically."
- The strategies differ in how they provide confirmation: the actual case strategy relies on frequency or magnitude of association across actual cases, while the counterfactual strategy cannot meaningfully assess frequencies and instead relies on general principles, theories, or laws distinct from the hypothesis being tested, plus historical facts relevant to the counterfactual scenario.
- Assessing relative causal weight in the counterfactual strategy requires a proliferation of counterfactual cases — to argue A was a more important cause than B, at least two scenarios are needed (A present/B absent versus B present/A absent).
Methods
- Conceptual and methodological argument developed in three sections: distinguishing the "actual case" and "counterfactual" strategies of hypothesis testing; illustrating the counterfactual strategy in practice through examples from international relations and comparative politics; and returning to theoretical issues about the link between causal arguments and counterfactual propositions.
- Illustrative examples include the "cult of the offensive" hypothesis about World War I (Van Evera 1984, Snyder 1984), structural versus domestic explanations of Soviet foreign policy (following Waltz 1979, 1981), and works such as Moore (1966), Skocpol (1979), and Luebbert (1987).
- No formal data collection or statistical analysis is reported; the paper is a methodological/theoretical essay.
Limitations
- The paper states that arguments about what would have happened in multiple counterfactual scenarios "will be very imprecise and uncertain," and that in the counterfactual strategy "there is no formal criterion for gauging the risk of error associated with some independent variable — all depends instead on the plausibility of arguments about 'what would have happened.'"
- The paper acknowledges that the counterfactual strategy for empirically checking a causal hypothesis "seems only indirectly empirical, since the confirmation it provides depends principally on other theories."
- Evident from the text: the provided full text is truncated mid-sentence in the second section ("Second, I will consider the role of counterfactuals in sever…"), so the paper's examples in practice and its third section on theoretical issues are only partially available here.
Model-written summary of a shortened version of the paper’s full text (it is longer than we read). Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 1 Not valid · 0 Not checked · 20 Own results and interpretations
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, even in N-1 case studies analysts will often mix both strategies of confirmation.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“work”
- Also stated in: § 5 For example: "The might-have-beens of history are not a profitable subject · plain
“in case studies where the analyst is explicitly concerned with giving a causal explanation for some event or phenomenom”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 5 For example: "The might-have-beens of history are not a profitable subject · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“causes”
- Also stated in: § 5 For example: "The might-have-beens of history are not a profitable subject · plain
“that selected the actual outcome from a range of possible counterfactual outcomes in that case, we see the source of the”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 5 For example: "The might-have-beens of history are not a profitable subject · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both strategies, if successful (from the analyst's point of view), tend to support the hypothesis that the proposed cause in fact produces (or produced) the effect.”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If there are”
- Also stated in: § 2 Following Waltz (1979, 1981), "structural position" here would entail the · plain
“other causes”
- Also stated in: § 2 Following Waltz (1979, 1981), "structural position" here would entail the · plain
“of the phenomenom in question that are not considered ®*P^icitly in the analysis, and if any of these are systematically related to the causes explicitly considered, then effects of the”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 2 Following Waltz (1979, 1981), "structural position" here would entail the · plain
7 more claims
Create a free account to see all 21 claims and every source verdict.
- Probably not a substantial claim. 3
- It may state two separate claims as one. 6
- The quote only partly states this claim. 2
Citations
Cites
Theory of International Politics
Frankel, J., & Waltz, K. N. (1980) · International Journal · doi:10.2307/40201892 · Cited by 8,506
Theory of International Politics, by Kenneth N. Waltz
Randle, R. F. (1980) · Political Science Quarterly · doi:10.2307/2149594 · Cited by 6,358
States and Social Revolutions: A Comparative Analysis of France, Russia and China
Skocpol, T. (1979) · Cambridge University Press · doi:10.1017/cbo9780511815805 · Cited by 5,408
Nagel, E., & Hawkins, D. (1961) · American Journal of Physics · doi:10.1119/1.1937571 · Cited by 3,667
Comparative Politics and the Comparative Method
Lijphart, A. (1971) · American Political Science Review · doi:10.2307/1955513 · Cited by 3,151
On the Social Origins of Dictatorship and Democracy
Moore, B. (1984) · Critique and Anti-Critique · doi:10.1007/978-1-349-86027-2_4 · Cited by 2,694
Methodology of Social Sciences
Weber, M. (2017) · Routledge eBooks · doi:10.4324/9781315124445 · Cited by 2,527
Greenstein, F. I., & Polsby, N. W. (1975) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 2,344
The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies
Ragin, C. C. (2014) · doi:10.1525/9780520957350 · Cited by 2,130
Rationalizable Strategic Behavior
Bernheim, B. D. (1984) · Econometrica · doi:10.2307/1911196 · Cited by 1,291
Research works printed but not linked: 8
- Ragin, 1987, The Comparative Method, 39
- George, 1985, Advances in Information Processing in Organizations, 2
- Gaddis John Lewis , “Nuclear Weapons and International Systemic Stability,” American Academy of Arts and Sciences Occasional Paper No. 2 (Cambridge: AAAS, 1990).
- Polsby, 1982, What If?: Essays in Social Science Fiction
- Moore, 1966, Social Origins of Dictatorship and Democracy, 414
- Fischer, 1970, Historians' Fallacies, 15
- Lewis, 1973, Counterfactuals, 65
- Barry, 1980, Supertax, Political Studies, 28, 139
Cited by
Items 701–725 of 725
Ferraz, J. (2025) · Revista Direito GV · doi:10.1590/2317-6172202535
International relations measurement
Muzas, B. K. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-26629-4.00209-4
Bell, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.5975597
Unfalsifiable Claims and Unaccountable Power: Why Theocracy Requires Thought ControlPreprint
Bell, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6109467
Barany, Z. D. (2026) · doi:10.1093/9780197679708.003.0002
Barany, Z. (2026) · doi:10.1093/9780197679708.005.0001
Barany, Z. (2026) · doi:10.1093/9780197679708.002.0004
Barany, Z. (2026) · doi:10.1093/9780197679708.003.0005
Barany, Z. D. (2026) · doi:10.1093/9780197679708.001.0001
Barany, Z. (2026) · doi:10.1093/9780197679708.003.0003
Barany, Z. (2026) · doi:10.1093/9780197679708.003.0004
Barany, Z. D. (2026) · doi:10.1093/9780197679708.003.0001
Barany, Z. (2026) · doi:10.1093/9780197679708.003.0006
Did the Cold War produce development clusters in Africa?
Dower, P. C., Gokmen, G., Breton, M. L., & Weber, S. (2026) · Journal of Development Economics · doi:10.1016/j.jdeveco.2026.103776
France–China cooperation on nuclear energy and artificial intelligence
Lee, Y. (2026) · French Politics · doi:10.1057/s41253-026-00312-y
Lai, C.-K. (2026) · The Extractive Industries and Society · doi:10.1016/j.exis.2026.101907
Towards a political anthropology of populism: historicality, relationality, modality
Kurt, F. (2026) · Third World Quarterly · doi:10.1080/01436597.2026.2646527
The End of Interdependence: Re-assessing Power, Progress, and Peace in a Fragmented World
Niyomana, V., & Mushitsi, P. (2026) · Journal of Social Sciences & Humanities · doi:10.62810/jssh.v3i2.292 · Cited by 1
“The methodological approach is deductive and theoretical, employing a structured counterfactual analysis — a tool in IR used to test the causal logic and robustness of core theories by examining a world in which a key variable (interdependence) is removed (Fearon, 1991; Tetlock & Lebow, 2001).”
InfluentialAmerican Exceptionalism in Comparative Perspective
Casas, J. (2026) · Studies in Comparative International Development · doi:10.1007/s12116-026-09493-7
“In a potential outcomes framework (Morgan and Winship 2014), counterfactuals are propositions that take the generic form “if it had been the case that C (or not C), it would have been the case that E (or not E)” (Fearon 1991, 169).”
Grynaviski, E. (2026) · doi:10.1093/9780197858974.003.0336
Thapa, I. (2026) · Research Square · doi:10.21203/rs.3.rs-10539830/v1
Walter, G., & Zürn, M. (2004) · Springer eBooks · doi:10.1007/978-1-4020-2208-1_12
Structured, Focused Uncertainty: Information Analysis for Multi-Method Comparative Case StudiesPreprint
Drozdova, K., & Gaubatz, K. T. (2009) · SSRN Electronic Journal
Metadata
- Authors
- Fearon, J. D.
- Affiliations
- University of California, Berkeley (US)
- Corresponding author
- Fearon, James D.
- Venue
- World Politics
- Publisher
- Project MUSE
- Published
- 1991-01
- Volume
- 43
- Issue
- 2
- Pages
- 169-195
- ISSN
- 0043-8871 (ISSN-L)
- 1086-3338
- OpenAlex
- W1972181371
- Semantic Scholar
- 17968a01ee432286912295946716661ac33e9b58
- Cited by
- 729
- References
- 74
- Citations by year
- 2026: 32 · 2025: 16 · 2024: 14 · 2023: 40 · 2022: 40 · 2021: 36 · 2020: 35 · 2019: 32 · 2018: 36 · 2017: 17 · 2016: 22 · 2015: 20 · 2014: 24 · 2013: 25 · 2012: 20
- FWCI
- 28.70
- Type
- journal article
- Language
- en
- Open access
- green
- escholarship.org
- Locations
- World Politics (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Virtual Defense Library (Ministerio de Defensa) (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Qualitative Comparative Analysis Research
- Social Sciences › Political Science and International Relations › Political Science Research and Education
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
- Keywords
- hypothesis testing, causal inference, comparative case study, small-N research, origins of World War I, social revolution, Latin America, origins of fascism, corporatism, international relations theory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


