The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index; 1 of the 65 sources its claims cite were checked for support.
Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference
Open accessAbstract
An increasing number of analyses in various subfields of political science employ Boolean algebra as proposed by Ragin's qualitative comparative analysis (QCA). This type of analysis is perfectly justifiable if the goal is to test deterministic hypotheses under the assumption of error-free measures of the employed variables. My contention is, however, that only in a very few research areas are our theories sufficiently advanced to yield deterministic hypotheses. Also, given the nature of our objects of study, error-free measures are largely an illusion. Hence, it is unsurprising that many studies employ QCA inductively and gloss over possible measurement errors. In this article, I address these issues and demonstrate the consequences of these problems with simple empirical examples. In an analysis similar to Monte Carlo simulation, I show that using Boolean algebra in an exploratory fashion without considering possible measurement errors may lead to dramatically misleading inferences. I then suggest remedies that help researchers to circumvent some of these pitfalls.
Overview
CoChat Summary
TL;DR
Hug argues that QCA is justified for testing deterministic hypotheses only when measures are error-free, but inductive use and measurement error can produce misleading inferences. Using welfare-state QCA data, he shows that dropping cases or recoding outcomes often changes the derived solutions, and he recommends tests for necessary conditions and caution against purely inductive QCA.
Key findings
- In a 16-case welfare-state QCA, dropping one observation left the original solution PUC + UCS in 14/16 (87.5%) runs; dropping two left it in 91/120 (75.83%) runs, with PUC and UCS each appearing in 14/120 (11.67%) and PUCS in 1/120 (0.83%).
- With one outcome recode, the original solution appeared in 12/16 (75%) runs; with two recodes, it appeared in slightly less than 40% of runs, and the probability of erroneous conclusions rose to more than 0.6. The paper states that when only slightly more than 5% of cases are affected by measurement error, there is already a 25% chance of reaching the wrong conclusion.
- In the necessary-condition example, strong left party as a necessary condition for strong welfare state had one counterexample among ten cases; the lower bound of a one-sided 95% confidence interval was 0.005. Assuming one case misclassified out of sixteen gives an error rate above 0.06, so the necessary-condition hypothesis cannot be rejected; estimated power was 0.405, implying type II error almost 0.6.
- Inductive use contradicts QCA’s original deductive design; without theoretical scope conditions, QCA yields only a description of the data at hand, and some solutions depend on single cases such as Ireland or Belgium.
Methods
- Conceptual critique plus a “seemingly” Monte Carlo simulation using existing QCA data from Ragin (2000), Grofman and Schneider (2009), and Skaaning (2011): 16 of 18 cases, four conditions (P, U, C, S), and outcome W.
- Replicated Grofman and Schneider’s QCA in R using Dusa’s implementation, then systematically dropped one or two observations to simulate misspecified scope and recoded the outcome for one or two observations to simulate measurement error; replication materials include a random Monte Carlo simulation with 1000 new data sets.
- Used Braumoeller and Goertz’s necessary-condition tests, including the pI-test, power/type II error assessment, and triviality test, alongside references to Dion and Clark, Gilligan, and Golder.
Limitations
- The paper states it deals only with measurement error in the dependent variable; consequences of measurement error in explanatory variables are likely harder to assess and correct for.
- The paper states there is no direct way to assess reliability of the measures in the example and assumes one case might be misclassified; it also notes that Braumoeller and Goertz’s tests address measurement error but not the inductive-use problem, and that avoiding inductivism seems more difficult.
- Evident from the text: the empirical demonstration is limited to one crisp-set QCA welfare-state dataset and systematic modifications; fuzzy-set QCA and other variants are discussed but not simulated, and the small number of cases prevented demonstrating asymmetric-hypothesis tests.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 9 Not valid · 51 Not checked · 86 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“while certain conditions may perhaps go hand in hand with a particular outcome (for instance, being registered in a course to get an A, to take Waldner's (2005) example), they hardly are causal explanations”
What the sources say
- Partly supportedcites Waldner (2005) sourceExperimental — accuracy not yet measured
On p. 28, Waldner argues that an observed association alone does not establish a causal relationship, supporting the sentence’s general point. But he uses course registration as an example of a necessary condition with an identified causal process—the administrative rule—not as a condition that merely goes hand in hand with an A. The passage does not address matching an inductive theory’s scope to the countries studied. (LLM, conf=0.84)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The necessarily deductive nature of this enterprise is often, however, neglected in practical research.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Careful reading of Ragin's (2000) book shows, however, that the fuzzy sets are not employed to allow for measurement error, but only allow for graded coding.”
What the sources say
- Not matchedcites Ragin 2000
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“Also, none of these three sets of authors consider the possibility that the variables they use might be affected by measurement error.”
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This short discussion of some prominent studies relying on QCA suggests that the necessary deductive underpinnings of causal paths are rarely offered.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
71 more claims
Create a free account to see all 147 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 56
- It may state two separate claims as one. 20
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The process of fuzzy-set scoring can result in measurement error, which must not be ignored, bringing the researcher to misleading inferences (Hug 2013: 252).”
From A qualitative assessment of QCA: method stretching in large-N studies and temporality (2022) · doi:10.1007/s11135-021-01278-5 · cites this paper as Hug 2013: 252
In the citing paper
“Allowing researchers to set both the bounds and the membership within them makes QCA intrinsically inductive to use and leads to problematic inferences (Hug 2013).”
From A qualitative assessment of QCA: method stretching in large-N studies and temporality (2022) · doi:10.1007/s11135-021-01278-5 · cites this paper as Hug 2013
In the citing paper
“Qualitative Comparative Analysis (QCA) is a descriptive research method that can provide causal explanations for an outcome of interest.”
From A qualitative assessment of QCA: method stretching in large-N studies and temporality (2022) · doi:10.1007/s11135-021-01278-5 · cites this paper as Hug 2013: 253
In the citing paper
“Scholars can use QCA productively for exploratory purposes for combinational causes of outcomes of interest (Berg-Schlosser et al. 2009: 17), descriptive analyses, or to test deterministic hypotheses (Hug 2013: 252).”
From A qualitative assessment of QCA: method stretching in large-N studies and temporality (2022) · doi:10.1007/s11135-021-01278-5 · cites this paper as Hug 2013: 252
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 6 Discussion and Conclusion
“31 This table also shows that assuming strong unions and sociocultural homogeneity to be necessary conditions for a strong welfare state would yield no counterexamples.”
§ 5 A Seemingly Monte Carlo Simulation
“| Belgium | 1 | 1 | 1 | 0 | 1 |”
Citations
Cites
The Logic of Scientific Discovery.
Hutchison, T. W., & Popper, K. R. (1959) · Economica · doi:10.2307/2550489 · Cited by 7,499
Designing Social Inquiry: Scientific Inference in Qualitative Research
King, G., Keohane, R. O., & Verba, S. (1994) · Princeton University Press · doi:10.1515/9781400821211 · Cited by 7,225
Redesigning Social Inquiry: Fuzzy Sets and Beyond
Ragin, C. C. (2008) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226702797.001.0001 · Cited by 4,445
Redesigning Social Inquiry: Fuzzy Sets and Beyond
Ragin, C. C. (2008) · Cited by 3,990
Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques
Rihoux, B., & Ragin, C. (2009) · SAGE Publications · doi:10.4135/9781452226569 · Cited by 3,089
Lieberson, S., & Ragin, C. C. (2001) · Contemporary Sociology · doi:10.2307/3089736 · Cited by 2,799
The Logic of Scientific Discovery
Popper, K. R., & Weiss, G. (1959) · Physics Today · doi:10.1063/1.3060577 · Cited by 2,717
Public Goods and Ethnic Divisions
Alesina, A., Baqir, R., & Easterly, W. (1999) · The Quarterly Journal of Economics · doi:10.1162/003355399556269 · Cited by 2,328
Public Goods and Ethnic Divisions
Alesina, A., Baqir, R., & Easterly, W. (1999) · Policy Research Working Papers · doi:10.1596/1813-9450-2108 · Cited by 2,179
The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies
Ragin, C. C. (2014) · doi:10.1525/9780520957350 · Cited by 2,130
Research works printed but not linked: 12
- Pearl, 2001, Causality
- Kittel, 2000, Der gezügelte Wohlfahrtsstaat: Sozialpolitik in Australien, Japan, Schweiz, Kanada, Neuseeland and den Vereinigten Staaten
- Gates Scott , and Strand Havard . 2004. Modeling the duration of civil wars: Measurement and estimation issues. Paper prepared for presentation at the Joint Session of Workshops of the ECPR, Uppsala.
- Dusa, 2006, QCA package for R version 0.1–3
- A series of recent review articles and books (e.g., Ragin, Berg-Schlosser, and de Meur 1996; Rihoux 2006a; Schneider and Wagemann 2007; Ragin 2008; Rihoux and Ragin 2008b) discuss the basic ideas stated in Ragin (1987, 2000) in more detail and offer “best practices” for users.
- Maggetti Martino , and Levi-Faur David . Forthcoming. Dealing with errors in QCA. Political Research Quarterly.
- Rihoux, 2008a, Configurational comparative methods: Qualitative comparative analysis (QCA) and related techniques, xvii
- Sekhon, 2005a, Probability tests require distribution, Qualitative Methods Newsletter, 3
- Mill, 1973, A system of logic, ratiocinative and inductive: Collected works of John Stuart Mill (Volume VII), 388
- George, 2005, Case studies and theory development
- Hug Simon . 2013. Replication data for Qualitative Comparative Analysis: How inductive use and measurement error lead to problematic inference. http://hdl.handle.net/1902.1/19489 IQSS Dataverse Network [Distributor] V1 [Version].
- Ragin (1987, 45) continues by stating that “[i]t is inductive because the investigator determines which of the theoretically relevant similarities and differences are operative by examining empirical cases.” His use of the term “inductive” is misleading, since what he describes is a simple process of theory-testing in Popper's (1959) sense, and certainly not in the way the latter author describes inductivism. Similarly, the end of Chapter 3 in Ragin (1987), while suggesting a back and forth between theory and empirics in QCA, nowhere states that QCA could be carried out without theory. This also implies that theory is needed to specify not only the components of a causal path, but also the latter's precise configuration.
Cited by
Designing Research With Qualitative Comparative Analysis (QCA): Approaches, Challenges, and Tools
(2017) · doi:10.1177/0049124117729700 · Cited by 380
“As Figure 1 illustrates, the mode of reasoning of a research study (upward) is intertwined with, but does not equate to, the question of how it relates to the underlying population (downward; contrary to, e.g., Hug 2013).”
BackgroundMethodInfluentialVis, B., & Dul, J. (2018) · Sociological Methods & Research · doi:10.1177/0049124115626179 · Cited by 298
“…for fuzzy set qualitative comparative analysis (fsQCA) than for necessary condition analysis, because some studies demonstrate that QCA—often of the crisp set variant— performs poorly in the presence of measurement error (Hug 2013; Krogslund, Choi, and Poertner 2015; Lucas and Szatrowski 2014).”
Qualitative Comparative Analysis Using R: A Beginner's Guide
Oana, I.-E., Schneider, C. Q., & Thomann, E. (2021) · doi:10.1017/9781009006781 · Cited by 224
QCA and business research: Work in progress or a consolidated agenda?
Wagemann, C., Buche, J., & Siewert, M. B. (2016) · Journal of Business Research · doi:10.1016/j.jbusres.2015.10.010 · Cited by 212
Often Trusted but Never (Properly) Tested: Evaluating Qualitative Comparative Analysis
Baumgartner, M., & Thiem, A. (2020) · Sociological Methods & Research · doi:10.1177/0049124117701487 · Cited by 193
“14 Lastly, critics have maintained that QCA solutions tend not to be robust (Hug 2013; Krogslund et al. 2015; Lucas and Szatrowski 2014).”
Background
Metadata
- Authors
- Hug, S.
- Affiliations
- University of Geneva (CH)
- Corresponding author
- Hug, Simon
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2013
- Volume
- 21
- Issue
- 2
- Pages
- 252-265
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W2142575249
- Cited by
- 137
- References
- 83
- Citations by year
- 2025: 6 · 2024: 4 · 2023: 7 · 2022: 6 · 2021: 21 · 2020: 25 · 2019: 7 · 2018: 12 · 2017: 12 · 2016: 10 · 2015: 15 · 2014: 12
- FWCI
- 36.64
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Political Analysis (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Archive ouverte UNIGE (University of Geneva) (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Qualitative Comparative Analysis Research
- Social Sciences › Sociology and Political Science › Political Conflict and Governance
- Social Sciences › Business, Management and Accounting › Strategy and Management › Political Influence and Corporate Strategies
- Keywords
- qualitative comparative analysis, Boolean algebra, measurement error, inductive reasoning, exploratory analysis, Monte Carlo simulation, causal inference, political science methodology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference. https://app.cochat.ai/papers/10.1093/pan/mps061
- The paper
- Hug, S. (2013). Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference. Political Analysis. https://doi.org/10.1093/pan/mps061
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.