Evidence and Inference in the Comparative Case Study
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Comparative case studies often rely on a practice known as selecting on the dependent variable. This technique involves choosing some phenomenon of political interest, gathering data on occurrences of the phenomenon, then determining what characteristics the occurrences have in common. A methodological strategy with a long history, selecting on the dependent variable has found numerous applications in political science, including the study of economic growth, social revolutions, and international conflict.' Selecting on the dependent variable has provided an important method for gathering information when there is relatively little data, though it is not limited to such situations. Its use has recently come under rather sharp attack. If observations are selected based on the value of the dependent variable, then estimates obtained by ordinary least squares will be biased. This bias can not be corrected by the introduction of control variables. Gathering more data will not solve the problem; the bias remains even if the number of observations goes to infinity. In short, inferences drawn from cases selected on the dependent variable are highly suspect.2 Impressive treatments of the problem from the position of comparative case study work have been presented by Barbara Geddes and by Gary King, Robert Keohane, and Sidney Verba.3 They have not only alerted the discipline to an important problem but have suggested concrete ways to deal with problems of selection bias.4 I would like to offer a qualified defense of analyses of a small number of cases selected on the dependent variable. As is well-known, selecting on the dependent variable is perfectly admissible if one is evaluating necessary (as opposed to sufficient) conditions.5 However, with a few exceptions, this argument has gone unnoticed in the debate on case selection.6 By concentrating on the logic of testing necessary conditions, this paper attempts to show that the methodological implications of selecting on the dependent variable are much less threatening to the validity of small-n comparative case study work than has often been claimed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These suggestions for explicitly (or implicitly) addressing measurement error made by Dion (1998), Braumoeller and Goertz (2000), and Clark, Gilligan, and Golder (2006) have, however, found little or any resonance among scholars employing QCA.”
From Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference (2013) · cites this paper as Dion (1998)
In the citing paper
“Dion (1998), in essence, adopts a Bayesian perspective where prior beliefs about necessary conditions are updated on the basis of new data according to Bayes's rule.”
From Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference (2013) · cites this paper as Dion (1998)
In the citing paper
“What remains unspecified, however, is where the uncertainty is coming from if some case(s) contradict(s) the necessary condition being tested.”
From Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference (2013) · cites this paper as Dion (1998)
In the citing paper
“This is also implied by Dion's (1998) finding that from a Bayesian perspective at least six cases are required to attain a sufficient level of certainty to maintain a necessary condition.”
From Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference (2013) · cites this paper as Dion (1998)
In the citing paper
“Dion (1998), in an award-winning article focusing in particular on necessary conditions, found using a Bayesian analysis that regularities can be established with con fi dence with as few as fi ve or six cases.”
From Large-N Qualitative Analysis (LNQA): Causal Generalization in Case Study and Multimethod Research (2023) · cites this paper as Dion (1998)
Citations
Cites
Capitalism and Social Democracy
Przeworski, A. (1987) · Labour / Le Travail · doi:10.2307/25142838 · Cited by 707
Without consent or contract: the rise and fall of American slavery
(1990) · Choice Reviews Online · doi:10.5860/choice.27-4712 · Cited by 533
Case Selection, Conceptualizations and Basic Logic in the Study of War
(1982) · doi:10.2307/2110974 · Cited by 25
Cited by
Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options
Seawright, J., & Gerring, J. (2008) · Political Research Quarterly · doi:10.1177/1065912907313077 · Cited by 3,827
What Is a Case Study and What Is It Good for?
GERRING, J. (2004) · American Political Science Review · doi:10.1017/s0003055404001182 · Cited by 2,995
A set-theoretic approach to organizational configurations
Fiss, P. C. (2007) · Academy of Management Review · doi:10.5465/amr.2007.26586092 · Cited by 2,225
“However, once the sample has been identified, selection on the dependent variable during the analysis is perfectly admissible to evaluate necessary conditions (Dion, 1998; Most & Starr, 1989).”
Set Relations in Social Research: Evaluating Their Consistency and Coverage
Ragin, C. C. (2006) · Political Analysis · doi:10.1093/pan/mpj019 · Cited by 1,784
Standards of Good Practice in Qualitative Comparative Analysis (QCA) and Fuzzy-Sets
Schneider, C. Q., & Wagemann, C. (2010) · Comparative Sociology · doi:10.1163/156913210x12493538729793 · Cited by 1,473
Dul, J. (2016) · Organizational Research Methods · doi:10.1177/1094428115584005 · Cited by 1,360
Case Studies: Types, Designs, and Logics of Inference
Levy, J. S. (2008) · Conflict Management and Peace Science · doi:10.1080/07388940701860318 · Cited by 1,050
A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research
Mahoney, J., & Goertz, G. (2006) · Political Analysis · doi:10.1093/pan/mpj017 · Cited by 904
Is There a (Viable) Crucial-Case Method?
Gerring, J. (2007) · Comparative Political Studies · doi:10.1177/0010414006290784 · Cited by 763
The Possibility Principle: Choosing Negative Cases in Comparative Research
MAHONEY, J., & GOERTZ, G. (2004) · American Political Science Review · doi:10.1017/s0003055404041401 · Cited by 615
A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences
Goertz, G., & Mahoney, J. (2012) · Cited by 582
QUALITATIVE RESEARCH: Recent Developments in Case Study Methods
Bennett, A., & Elman, C. (2006) · Annual Review of Political Science · doi:10.1146/annurev.polisci.8.082103.104918 · Cited by 577
War and Peace in International Rivalry
Diehl, P., & Goertz, G. (2000) · University of Michigan Press eBooks · doi:10.3998/mpub.16693 · Cited by 504
Anckar, C. (2008) · International Journal of Social Research Methodology · doi:10.1080/13645570701401552 · Cited by 472
Rihoux, B. (2006) · International Sociology · doi:10.1177/0268580906067836 · Cited by 461
Presidential Impeachment and the New Political Instability in Latin America
Pérez-Liñán, A. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511510335 · Cited by 442
Case Selection for Case‐Study Analysis: Qualitative and Quantitative Techniques
Gerring, J. (2009) · The Oxford Handbook of Political Methodology · doi:10.1093/oxfordhb/9780199286546.003.0028 · Cited by 414
Engle, R. A. (2006) · Journal of the Learning Sciences · doi:10.1207/s15327809jls1504_2 · Cited by 395
Anderson, E. (2004) · Hypatia · doi:10.1111/j.1527-2001.2004.tb01266.x · Cited by 391
Comparative Democratization: Big and Bounded Generalizations
Bunce, V. (2000) · Comparative Political Studies · doi:10.1177/001041400003300602 · Cited by 353
Comparative-Historical Methodology
Mahoney, J. (2004) · Annual Review of Sociology · doi:10.1146/annurev.soc.30.012703.110507 · Cited by 347
Toward a Unified Theory of Causality
Mahoney, J. (2008) · Comparative Political Studies · doi:10.1177/0010414007313115 · Cited by 343
The Enduring Indispensability of the Controlled Comparison
Slater, D., & Ziblatt, D. (2013) · Comparative Political Studies · doi:10.1177/0010414012472469 · Cited by 332
Qualitative Methodology and Comparative Politics
Mahoney, J. (2007) · Comparative Political Studies · doi:10.1177/0010414006296345 · Cited by 320
CAN ONE OR A FEW CASES YIELD THEORETICAL GAINS?
Rueschemeyer, D. (2003) · Comparative Historical Analysis in the Social Sciences · doi:10.1017/cbo9780511803963.010 · Cited by 321
Strategies of Causal Inference in Small-N Analysis
MAHONEY, J. (2000) · Sociological Methods & Research · doi:10.1177/0049124100028004001 · Cited by 302
The Logic of Historical Explanation in the Social Sciences
Mahoney, J., Kimball, E., & Koivu, K. L. (2009) · Comparative Political Studies · doi:10.1177/0010414008325433 · Cited by 301
Democratization and Research Methods
Coppedge, M. (2012) · doi:10.1017/cbo9781139016179 · Cited by 303
Selecting Cases for Intensive Analysis: A Diversity of Goals and Methods
Gerring, J., & Cojocaru, L. (2016) · Sociological Methods & Research · doi:10.1177/0049124116631692 · Cited by 298
The Correlates of Nuclear Proliferation: A Quantitative Test
Singh, S., & Way, C. R. (2004) · Journal of Conflict Resolution · doi:10.1177/0022002704269655 · Cited by 277
Gerring, J. (2010) · Comparative Political Studies · doi:10.1177/0010414010376911 · Cited by 279
Schneider, C. Q., & Wagemann, C. (2006) · European Journal of Political Research · doi:10.1111/j.1475-6765.2006.00635.x · Cited by 277
Narrative Explanation: An Alternative to Variable-Centered Explanation?
Abell, P. (2004) · Annual Review of Sociology · doi:10.1146/annurev.soc.29.010202.100113 · Cited by 264
COMPARATIVE HISTORICAL ANALYSIS: ACHIEVEMENTS AND AGENDAS
Mahoney, J., & Rueschemeyer, D. (2003) · Comparative Historical Analysis in the Social Sciences · doi:10.1017/cbo9780511803963.002 · Cited by 262
Strategies of Causal Assessment in Comparative Historical Analysis
Mahoney, J. (2003) · Comparative Historical Analysis in the Social Sciences · doi:10.1017/cbo9780511803963.011 · Cited by 260
When Less is More: Selection Problems in Large- N and Small- N Cross-National Comparisons
Ebbinghaus, B. (2005) · International Sociology · doi:10.1177/0268580905052366 · Cited by 201
“Using, for instance, a prior probability of 50 percent that the alternative hypothesis is correct, only five confirming cases are needed to reach a 95 percent confidence that a condition is indeed necessary (Dion, 1998).”
A Pragmatic Guide to Qualitative Historical Analysis in the Study of International Relations
Thies, C. G. (2002) · International Studies Perspectives · doi:10.1111/1528-3577.t01-1-00099 · Cited by 194
An Introduction to Crisp Set QCA, with a Comparison to Binary Logistic Regression
Grofman, B., & Schneider, C. Q. (2009) · Political Research Quarterly · doi:10.1177/1065912909338464 · Cited by 192
Long‐Run Development and the Legacy of Colonialism in Spanish America
Mahoney, J. (2003) · American Journal of Sociology · doi:10.1086/378454 · Cited by 185
The Case Study: What it is and What it Does
Gerring, J. (2009) · The Oxford Handbook of Comparative Politics · doi:10.1093/oxfordhb/9780199566020.003.0004 · Cited by 189
Causation: A Unified Framework for the Social Sciences
Gerring, J. (2005) · Journal of Theoretical Politics · doi:10.1177/0951629805050859 · Cited by 189
Poteete, A. R., & Ostrom, E. (2008) · World Development · doi:10.1016/j.worlddev.2007.02.012 · Cited by 184
Opposites Attract? Opportunities and Challenges for Integrating Large-N QCA and Econometric Analysis
Fiss, P. C., Sharapov, D., & Cronqvist, L. (2013) · Political Research Quarterly · Cited by 262
Causal Complexity and the Study of Politics
Braumoeller, B. F. (2003) · Political Analysis · doi:10.1093/pan/mpg012 · Cited by 179
Blatt, R. (2009) · Academy of Management Review · doi:10.5465/amr.2009.40633298 · Cited by 178
QCA, 25 Years after “The Comparative Method”: Mapping, Challenges, and Innovations—Mini-Symposium
Rihoux, B., & Marx, A. (2013) · Political Research Quarterly · doi:10.1177/1065912912468269 · Cited by 194
Walker, C., & Baxter, J. (2017) · Energy Research & Social Science · doi:10.1016/j.erss.2017.05.016 · Cited by 179
A Future for Organization Theory: Living in and Living with Changing Organizations
Walsh, J. P., Meyer, A. D., & Schoonhoven, C. B. (2006) · Organization Science · doi:10.1287/orsc.1060.0215 · Cited by 194
Harding, D. J., Fox, C., & Mehta, J. D. (2002) · Sociological Methods & Research · doi:10.1177/0049124102031002003 · Cited by 194
Two-Level Theories and Fuzzy-Set Analysis
Goertz, G., & Mahoney, J. (2005) · Sociological Methods & Research · doi:10.1177/0049124104266128 · Cited by 176
Slater, D., & Ziblatt, D. (1982) · Electoral Studies · doi:10.1016/0261-3794(82)90242-6 · Cited by 149
Elite Interviews and State Politics Research
Beamer, G. (2002) · State Politics & Policy Quarterly · doi:10.1177/153244000200200106 · Cited by 245
Hug, S. (2013) · Political Analysis · doi:10.1093/pan/mps061 · Cited by 137
Selection Bias in Comparative Research: The Case of Incomplete Data Sets
Hug, S. (2003) · Political Analysis · doi:10.1093/pan/mpg014 · Cited by 154
The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes
Schuurman, D., Marez, L. D., & Ballon, P. (2016) · Technology Innovation Management Review · doi:10.22215/timreview/956 · Cited by 213
Goertz, G. (2006) · Studies in Comparative International Development · doi:10.1007/bf02686312 · Cited by 170
The powers of the Union: delegation in the EU
(2008) · Choice Reviews Online · doi:10.5860/choice.45-4622 · Cited by 131
A Simple Multivariate Test for Asymmetric Hypotheses
Clark, W. R., Gilligan, M. J., & Golder, M. (2006) · Political Analysis · doi:10.1093/pan/mpj018 · Cited by 120
The Moderation Theory Revisited: The Case of Islamic Political Actors
Tezcür, G. M. (2009) · Party Politics · doi:10.1177/1354068809339536 · Cited by 148
From Methodology to Practice: Inductive Iteration in Comparative Research
Yom, S. (2015) · Comparative Political Studies · doi:10.1177/0010414014554685 · Cited by 110
Is There a Necessary Condition for Democracy? The Role of State Capacity in Postcommunist Countries
Fortin‐Rittberger, J. (2011) · Comparative Political Studies · doi:10.1177/0010414011428587 · Cited by 148
Diasporas and conflict: distance, contiguity and spheres of engagement
van Hear, N., & Cohen, R. (2017) · Oxford Development Studies · doi:10.1080/13600818.2016.1160043 · Cited by 76
Preventive War and Democratic Politics.
Levy, J. S. (2008) · International Studies Quarterly · doi:10.1111/j.1468-2478.2007.00489.x · Cited by 106
Karapin, R. (1999) · Comparative Politics · doi:10.2307/422238 · Cited by 103
Quality Meets Quantity: Case Studies, Conditional Probability, and Counterfactuals
Sekhon, J. S. (2004) · Perspectives on Politics · doi:10.1017/s1537592704040150 · Cited by 97
Nielsen, R. A. (2014) · Sociological Methods & Research · doi:10.1177/0049124114547054 · Cited by 121
Necessary condition hypotheses in operations management
Dul, J., Hak, T., Goertz, G., & Voss, C. (2010) · International Journal of Operations & Production Management · doi:10.1108/01443571011087378 · Cited by 93
Canons of research design in qualitative analysis
Munck, G. L. (1998) · Studies in Comparative International Development · doi:10.1007/bf02687490 · Cited by 113
Testing for Necessary and/or Sufficient Causation: Which Cases Are Relevant?
Seawright, J. (2002) · Political Analysis · doi:10.1093/pan/10.2.178 · Cited by 85
School Shootings as Organizational Deviance
Fox, C., & Harding, D. J. (2005) · Sociology of Education · doi:10.1177/003804070507800104 · Cited by 104
Ripsman, N. M. (2005) · International Studies Quarterly · doi:10.1111/j.1468-2478.2005.00383.x · Cited by 86
Strategies for Studying Causation in Complex Ecological-Political Systems
Homer-Dixon, T. F. (1996) · The Journal of Environment & Development · doi:10.1177/107049659600500202 · Cited by 79
Guarding Against False Positives in Qualitative Comparative Analysis
Braumoeller, B. F. (2015) · Political Analysis · doi:10.1093/pan/mpv017 · Cited by 77
Bennett, A. J., & Elman, C. (2009) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199219322.003.0029 · Cited by 87
Influences of media on social movements: Problematizing hyperbolic inferences about impacts
Olorunnisola, A. A., & Martin, B. L. (2012) · Telematics and Informatics · doi:10.1016/j.tele.2012.02.005 · Cited by 98
Disciplinary Logics in Doctoral Admissions: Understanding Patterns of Faculty Evaluation
Posselt, J. R. (2015) · The Journal of Higher Education · doi:10.1080/00221546.2015.11777385 · Cited by 61
A Two-Level Theory of Presidential Instability
Pérez‐Liñán, A. (2014) · Latin American Politics and Society · doi:10.1111/j.1548-2456.2014.00220.x · Cited by 88
Direct Democracy in Microstates and Small Island States
Anckar, D. (2003) · World Development · doi:10.1016/j.worlddev.2003.08.003 · Cited by 67
Dul, J. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2588480 · Cited by 57
Comparing blunders in government
Jennings, W., Lodge, M., & Ryan, M. E. (2017) · European Journal of Political Research · doi:10.1111/1475-6765.12230 · Cited by 53
Overcoming the fear of commitment: Pre-electoral coalitions in Norway and Sweden
Allern, E. H., & Aylott, N. (2009) · Acta Politica · doi:10.1057/ap.2009.1 · Cited by 48
Historical Enquiry and Comparative Politics
Mahoney, J., & Villegas, C. M. (2009) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199566020.003.0003 · Cited by 47
Bakke, K. M., Cao, X., Ο’Loughlin, J., & Ward, M. D. (2009) · Nations and Nationalism · doi:10.1111/j.1469-8129.2009.00363.x · Cited by 47
Qualitative Methods and Cross-Method Dialogue in Political Science
Levy, J. S. (2007) · Comparative Political Studies · doi:10.1177/0010414006296348 · Cited by 51
Case selection and causal inferences in qualitative comparative research
Plümper, T., Troeger, V. E., & Neumayer, E. (2019) · PLoS ONE · doi:10.1371/journal.pone.0219727 · Cited by 66
Paired Comparison and Theory Development: Considerations for Case Selection
Gisselquist, R. M. (2014) · PS Political Science & Politics · doi:10.1017/s1049096514000419 · Cited by 50
Comparative-Historical Analysis in Contemporary Political Science
Mahoney, J., & Terrie, L. (2009) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199286546.003.0032 · Cited by 43
Interplay of land governance and large-scale agricultural investment: evidence from Ghana and Kenya
Nolte, K., & Väth, S. J. (2015) · The Journal of Modern African Studies · doi:10.1017/s0022278x14000688 · Cited by 47
Buyer–supplier interaction in business-to-business services: A typology test using case research
van der Valk, W., & Wynstra, F. (2012) · Journal of Purchasing and Supply Management · doi:10.1016/j.pursup.2012.03.003 · Cited by 39
Buyer-seller interaction patterns during ongoing service exchange
van der Valk, W. (2007) · TU/e Research Portal · doi:10.6100/ir806189 · Cited by 103
Transnational Labour Campaigns: Can the Logic of the Market Be Turned Against Itself?
Seidman, G. W. (2008) · Development and Change · doi:10.1111/j.1467-7660.2008.00525.x · Cited by 36
Engle, R. A., & Faux, R. B. (2006) · Cited by 57
Cardinale, R. (2019) · Energy Policy · doi:10.1016/j.enpol.2019.03.040 · Cited by 34
How Good Is Good Enough? A Multidimensional, Best-Possible Standard for Research Design
Gerring, J. (2010) · Political Research Quarterly · doi:10.1177/1065912910361221 · Cited by 63
Ontology, Epistemology, and Multimethod Research in Political Science
Chatterjee, A. (2011) · Philosophy of the Social Sciences · doi:10.1177/0048393111415380 · Cited by 39
Standards guter Praxis in Qualitative Comparative Analysis (QCA) und Fuzzy-Sets
Schneider, C. Q., & Wagemann, C. (2009) · VS Verlag für Sozialwissenschaften eBooks · doi:10.1007/978-3-531-91826-6_19 · Cited by 43
When less is more: selection problems in large-N and small-N cross-national comparisons
Ebbinghaus, B. (2006) · MADOC (University of Mannheim) · Cited by 31
Goerres, A., Siewert, M. Β., & Wagemann, C. (2019) · KZfSS Kölner Zeitschrift für Soziologie und Sozialpsychologie · doi:10.1007/s11577-019-00600-2 · Cited by 35
Metadata
- Authors
- Dion, D.
- Corresponding author
- Dion, Douglas
- Venue
- Comparative Politics
- Publisher
- JSTOR
- Published
- 1998-01
- Volume
- 30
- Issue
- 2
- Pages
- 127
- ISSN
- 0010-4159 (ISSN-L)
- OpenAlex
- W2017659746
- Cited by
- 261
- References
- 3
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 2 · 2023: 10 · 2022: 2 · 2021: 1 · 2020: 7 · 2019: 9 · 2018: 7 · 2017: 11 · 2016: 7 · 2015: 12 · 2014: 10 · 2013: 12 · 2012: 13
- FWCI
- 20.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Comparative Politics (publishedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Qualitative Comparative Analysis Research
- Social Sciences › Political Science and International Relations › Political Science Research and Education
- Social Sciences › Sociology and Political Science › Political Conflict and Governance
- Keywords
- selection bias, comparative case study, necessary conditions, sufficient conditions, small-N research, ordinary least squares bias, case selection, validity of inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.