Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference
Open accessThe DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index; 1 of the 65 sources its claims cite were checked for support.
Abstract
An increasing number of analyses in various subfields of political science employ Boolean algebra as proposed by Ragin's qualitative comparative analysis (QCA). This type of analysis is perfectly justifiable if the goal is to test deterministic hypotheses under the assumption of error-free measures of the employed variables. My contention is, however, that only in a very few research areas are our theories sufficiently advanced to yield deterministic hypotheses. Also, given the nature of our objects of study, error-free measures are largely an illusion. Hence, it is unsurprising that many studies employ QCA inductively and gloss over possible measurement errors. In this article, I address these issues and demonstrate the consequences of these problems with simple empirical examples. In an analysis similar to Monte Carlo simulation, I show that using Boolean algebra in an exploratory fashion without considering possible measurement errors may lead to dramatically misleading inferences. I then suggest remedies that help researchers to circumvent some of these pitfalls.
Overview
CoChat Summary
TL;DR
Hug argues that QCA is justified for testing deterministic hypotheses only when measures are error-free, but inductive use and measurement error can produce misleading inferences. Using welfare-state QCA data, he shows that dropping cases or recoding outcomes often changes the derived solutions, and he recommends tests for necessary conditions and caution against purely inductive QCA.
Key findings
- In a 16-case welfare-state QCA, dropping one observation left the original solution PUC + UCS in 14/16 (87.5%) runs; dropping two left it in 91/120 (75.83%) runs, with PUC and UCS each appearing in 14/120 (11.67%) and PUCS in 1/120 (0.83%).
- With one outcome recode, the original solution appeared in 12/16 (75%) runs; with two recodes, it appeared in slightly less than 40% of runs, and the probability of erroneous conclusions rose to more than 0.6. The paper states that when only slightly more than 5% of cases are affected by measurement error, there is already a 25% chance of reaching the wrong conclusion.
- In the necessary-condition example, strong left party as a necessary condition for strong welfare state had one counterexample among ten cases; the lower bound of a one-sided 95% confidence interval was 0.005. Assuming one case misclassified out of sixteen gives an error rate above 0.06, so the necessary-condition hypothesis cannot be rejected; estimated power was 0.405, implying type II error almost 0.6.
- Inductive use contradicts QCA’s original deductive design; without theoretical scope conditions, QCA yields only a description of the data at hand, and some solutions depend on single cases such as Ireland or Belgium.
Methods
- Conceptual critique plus a “seemingly” Monte Carlo simulation using existing QCA data from Ragin (2000), Grofman and Schneider (2009), and Skaaning (2011): 16 of 18 cases, four conditions (P, U, C, S), and outcome W.
- Replicated Grofman and Schneider’s QCA in R using Dusa’s implementation, then systematically dropped one or two observations to simulate misspecified scope and recoded the outcome for one or two observations to simulate measurement error; replication materials include a random Monte Carlo simulation with 1000 new data sets.
- Used Braumoeller and Goertz’s necessary-condition tests, including the pI-test, power/type II error assessment, and triviality test, alongside references to Dion and Clark, Gilligan, and Golder.
Limitations
- The paper states it deals only with measurement error in the dependent variable; consequences of measurement error in explanatory variables are likely harder to assess and correct for.
- The paper states there is no direct way to assess reliability of the measures in the example and assumes one case might be misclassified; it also notes that Braumoeller and Goertz’s tests address measurement error but not the inductive-use problem, and that avoiding inductivism seems more difficult.
- Evident from the text: the empirical demonstration is limited to one crisp-set QCA welfare-state dataset and systematic modifications; fuzzy-set QCA and other variants are discussed but not simulated, and the small number of cases prevented demonstrating asymmetric-hypothesis tests.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 9 Not valid · 51 Not checked · 86 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“while certain conditions may perhaps go hand in hand with a particular outcome (for instance, being registered in a course to get an A, to take Waldner's (2005) example), they hardly are causal explanations”
What the sources say
- Partly supportedcites Waldner (2005) sourceExperimental — accuracy not yet measured
On p. 28, Waldner argues that an observed association alone does not establish a causal relationship, supporting the sentence’s general point. But he uses course registration as an example of a necessary condition with an identified causal process—the administrative rule—not as a condition that merely goes hand in hand with an A. The passage does not address matching an inductive theory’s scope to the countries studied. (LLM, conf=0.84)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The necessarily deductive nature of this enterprise is often, however, neglected in practical research.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Careful reading of Ragin's (2000) book shows, however, that the fuzzy sets are not employed to allow for measurement error, but only allow for graded coding.”
What the sources say
- Not matchedcites Ragin 2000
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“Also, none of these three sets of authors consider the possibility that the variables they use might be affected by measurement error.”
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This short discussion of some prominent studies relying on QCA suggests that the necessary deductive underpinnings of causal paths are rarely offered.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
71 more claims
Create a free account to see all 147 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 56
- It may state two separate claims as one. 20
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The process of fuzzy-set scoring can result in measurement error, which must not be ignored, bringing the researcher to misleading inferences (Hug 2013: 252).”
From A qualitative assessment of QCA: method stretching in large-N studies and temporality (2022) · doi:10.1007/s11135-021-01278-5 · cites this paper as Hug 2013: 252
In the citing paper
“Allowing researchers to set both the bounds and the membership within them makes QCA intrinsically inductive to use and leads to problematic inferences (Hug 2013).”
From A qualitative assessment of QCA: method stretching in large-N studies and temporality (2022) · doi:10.1007/s11135-021-01278-5 · cites this paper as Hug 2013
In the citing paper
“Qualitative Comparative Analysis (QCA) is a descriptive research method that can provide causal explanations for an outcome of interest.”
From A qualitative assessment of QCA: method stretching in large-N studies and temporality (2022) · doi:10.1007/s11135-021-01278-5 · cites this paper as Hug 2013: 253
In the citing paper
“Scholars can use QCA productively for exploratory purposes for combinational causes of outcomes of interest (Berg-Schlosser et al. 2009: 17), descriptive analyses, or to test deterministic hypotheses (Hug 2013: 252).”
From A qualitative assessment of QCA: method stretching in large-N studies and temporality (2022) · doi:10.1007/s11135-021-01278-5 · cites this paper as Hug 2013: 252
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 6 Discussion and Conclusion
“31 This table also shows that assuming strong unions and sociocultural homogeneity to be necessary conditions for a strong welfare state would yield no counterexamples.”
§ 5 A Seemingly Monte Carlo Simulation
“| Belgium | 1 | 1 | 1 | 0 | 1 |”
Citations
Cites
The Logic of Scientific Discovery.
Hutchison, T. W., & Popper, K. R. (1959) · Economica · doi:10.2307/2550489 · Cited by 7,499
Designing Social Inquiry: Scientific Inference in Qualitative Research
King, G., Keohane, R. O., & Verba, S. (1994) · Princeton University Press · doi:10.1515/9781400821211 · Cited by 7,225
Redesigning Social Inquiry: Fuzzy Sets and Beyond
Ragin, C. C. (2008) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226702797.001.0001 · Cited by 4,445
Redesigning Social Inquiry: Fuzzy Sets and Beyond
Ragin, C. C. (2008) · Cited by 3,990
Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques
Rihoux, B., & Ragin, C. (2009) · SAGE Publications · doi:10.4135/9781452226569 · Cited by 3,089
Lieberson, S., & Ragin, C. C. (2001) · Contemporary Sociology · doi:10.2307/3089736 · Cited by 2,799
The Logic of Scientific Discovery
Popper, K. R., & Weiss, G. (1959) · Physics Today · doi:10.1063/1.3060577 · Cited by 2,717
Public Goods and Ethnic Divisions
Alesina, A., Baqir, R., & Easterly, W. (1999) · The Quarterly Journal of Economics · doi:10.1162/003355399556269 · Cited by 2,328
Public Goods and Ethnic Divisions
Alesina, A., Baqir, R., & Easterly, W. (1999) · Policy Research Working Papers · doi:10.1596/1813-9450-2108 · Cited by 2,179
The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies
Ragin, C. C. (2014) · doi:10.1525/9780520957350 · Cited by 2,130
Research works printed but not linked: 12
- Pearl, 2001, Causality
- Kittel, 2000, Der gezügelte Wohlfahrtsstaat: Sozialpolitik in Australien, Japan, Schweiz, Kanada, Neuseeland and den Vereinigten Staaten
- Gates Scott , and Strand Havard . 2004. Modeling the duration of civil wars: Measurement and estimation issues. Paper prepared for presentation at the Joint Session of Workshops of the ECPR, Uppsala.
- Dusa, 2006, QCA package for R version 0.1–3
- A series of recent review articles and books (e.g., Ragin, Berg-Schlosser, and de Meur 1996; Rihoux 2006a; Schneider and Wagemann 2007; Ragin 2008; Rihoux and Ragin 2008b) discuss the basic ideas stated in Ragin (1987, 2000) in more detail and offer “best practices” for users.
- Maggetti Martino , and Levi-Faur David . Forthcoming. Dealing with errors in QCA. Political Research Quarterly.
- Rihoux, 2008a, Configurational comparative methods: Qualitative comparative analysis (QCA) and related techniques, xvii
- Sekhon, 2005a, Probability tests require distribution, Qualitative Methods Newsletter, 3
- Mill, 1973, A system of logic, ratiocinative and inductive: Collected works of John Stuart Mill (Volume VII), 388
- George, 2005, Case studies and theory development
- Hug Simon . 2013. Replication data for Qualitative Comparative Analysis: How inductive use and measurement error lead to problematic inference. http://hdl.handle.net/1902.1/19489 IQSS Dataverse Network [Distributor] V1 [Version].
- Ragin (1987, 45) continues by stating that “[i]t is inductive because the investigator determines which of the theoretically relevant similarities and differences are operative by examining empirical cases.” His use of the term “inductive” is misleading, since what he describes is a simple process of theory-testing in Popper's (1959) sense, and certainly not in the way the latter author describes inductivism. Similarly, the end of Chapter 3 in Ragin (1987), while suggesting a back and forth between theory and empirics in QCA, nowhere states that QCA could be carried out without theory. This also implies that theory is needed to specify not only the components of a causal path, but also the latter's precise configuration.
Cited by
Designing Research With Qualitative Comparative Analysis (QCA): Approaches, Challenges, and Tools
(2017) · doi:10.1177/0049124117729700 · Cited by 380
“As Figure 1 illustrates, the mode of reasoning of a research study (upward) is intertwined with, but does not equate to, the question of how it relates to the underlying population (downward; contrary to, e.g., Hug 2013).”
BackgroundMethodInfluentialVis, B., & Dul, J. (2018) · Sociological Methods & Research · doi:10.1177/0049124115626179 · Cited by 298
“…for fuzzy set qualitative comparative analysis (fsQCA) than for necessary condition analysis, because some studies demonstrate that QCA—often of the crisp set variant— performs poorly in the presence of measurement error (Hug 2013; Krogslund, Choi, and Poertner 2015; Lucas and Szatrowski 2014).”
Qualitative Comparative Analysis Using R: A Beginner's Guide
Oana, I.-E., Schneider, C. Q., & Thomann, E. (2021) · doi:10.1017/9781009006781 · Cited by 224
QCA and business research: Work in progress or a consolidated agenda?
Wagemann, C., Buche, J., & Siewert, M. B. (2016) · Journal of Business Research · doi:10.1016/j.jbusres.2015.10.010 · Cited by 212
Often Trusted but Never (Properly) Tested: Evaluating Qualitative Comparative Analysis
Baumgartner, M., & Thiem, A. (2020) · Sociological Methods & Research · doi:10.1177/0049124117701487 · Cited by 193
“14 Lastly, critics have maintained that QCA solutions tend not to be robust (Hug 2013; Krogslund et al. 2015; Lucas and Szatrowski 2014).”
BackgroundA Reviewer’s Guide to Qualitative Rigor
Nowell, B., & Albrecht, K. (2019) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muy052 · Cited by 185
A General Approach to Panel Data Set-Theoretic Research
Castro, R. G., & Ariño, M. A. (2016) · Journal of Advances in Management Sciences & Information Systems · doi:10.6000/2371-1647.2016.02.06 · Cited by 183
“Previous research has shown that robustness tests are indispensable in set-theoretic research (Hug 2013 [36]; Schneider and Wagemann 2012 [19]; Skaaning 2011 [37]; Thiem 2013 [38]).”
A Robustness Test Protocol for Applied QCA: Theory and R Software Application
Oana, I., & Schneider, C. Q. (2021) · Sociological Methods & Research · doi:10.1177/00491241211036158 · Cited by 184
Thiem, A., Baumgartner, M., & Bol, D. (2016) · Comparative Political Studies · doi:10.1177/0010414014565892 · Cited by 138
“…because RAMs with interactions offer a superior means for testing (probabilistic) hypotheses about necessity and sufficiency relations, a view that appears to have recently convinced a number of scholars (Brady, 2013, pp. 258-263; Fiss, Sharapov, & Cronqvist, 2013, p. 192; Hug, 2013, p. 257).”
BackgroundMethodFuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA
Krogslund, C., Choi, D. D., & Poertner, M. (2015) · Political Analysis · doi:10.1093/pan/mpu016 · Cited by 136
Why do conflicts over scarce renewable resources turn violent? A qualitative comparative analysis
Ide, T. (2015) · Global Environmental Change · doi:10.1016/j.gloenvcha.2015.04.008 · Cited by 134
Applying and Assessing Large-N QCA: Causality and Robustness From a Critical Realist Perspective
Rutten, R. (2020) · Sociological Methods & Research · doi:10.1177/0049124120914955 · Cited by 140
“Higher consistency thresholds also reduce the number of inconsistent cases, which is helpful because the absolute number of 18 Sociological Methods & Research XX(X) inconsistent cases affects consistency by defeating the implied cross-case regularity (Hug 2013:263).”
BackgroundMethodChallenging trends in configuration research: Where are the configurations?
Miller, D. (2018) · Strategic Organization · doi:10.1177/1476127017729315 · Cited by 112
“…a variety of attribute cutoffs (see the review by Misangyi et al., 2017), but in instances where this is not done thoroughly, study findings are vulnerable to small alterations in attribute sets and cutoffs (see Hug, 2013 and Krogslund et al., 2015 regarding susceptibility of QCA to type 1 error).”
Backgroundde Block, D., & Vis, B. (2018) · Journal of Mixed Methods Research · doi:10.1177/1558689818770061 · Cited by 135
“In addition, the literature criticizing QCA (e.g., Hug, 2013; Lucas & Szatrowski, 2014; Paine, 2016) regularly indicates that the alleged lack of findings’ robustness is a key problem (but see Baumgartner & Thiem, 2017b).”
BackgroundSociology as a Population Science
Goldthorpe, J. H. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781316412565 · Cited by 174
Qualitative Comparative Analysis and the Study of Policy Processes
Fischer, M., & Maggetti, M. (2017) · Journal of Comparative Policy Analysis: Research and Practice · doi:10.1080/13876988.2016.1149281 · Cited by 96
“controversially discusses issues of measurement error or the robustness of QCA results when changing the calibration or thresholds (Skaaning 2011; Hug 2013; Maggetti and Levi-Faur 2013; Fiss et al. 2014; Lucas and Szatrowski 2014, Thiem 2014; Vaisey 2014; Braumoeller 2015; Krogslund et al. 2015; Thiem et al. 2015).”
BackgroundInfluentialResearch methods for exploring the links between climate change and conflict
Ide, T. (2017) · WIREs Climate Change · doi:10.1002/wcc.456 · Cited by 95
Cooper, B., & Glaesser, J. (2016) · International Journal of Social Research Methodology · doi:10.1080/13645579.2015.1033799 · Cited by 94
“Previous robustness studies (Hug, 2013; Krogslund & Michel, 2014; Lucas & Szatrowski, 2014; Schneider & Wagemann, 2012; Seawright, 2005; Skaaning, 2011) have either focused on the effects of choices scholars must make in calibrating sets, in choosing consistency thresholds for…”
BackgroundDeclaring and Diagnosing Research Designs
BLAIR, G., COOPER, J., COPPOCK, A., & HUMPHREYS, M. (2019) · American Political Science Review · doi:10.1017/s0003055419000194 · Cited by 99
Kirchherr, J., Charles, K., & Walton, M. J. (2016) · Global Environmental Change · doi:10.1016/j.gloenvcha.2016.08.001 · Cited by 84
Opposition “overblown”? Community response to wind energy siting in the Western United States
Giordono, L. S., Boudet, H. S., Karmazina, A., Taylor, C. L., & Steel, B. S. (2018) · Energy Research & Social Science · doi:10.1016/j.erss.2018.05.016 · Cited by 92
Guarding Against False Positives in Qualitative Comparative Analysis
Braumoeller, B. F. (2015) · Political Analysis · doi:10.1093/pan/mpv017 · Cited by 77
“Moreover, as Hug (2013) points out, csQCA’s sensitivity to counterexamples makes it vulnerable to even a small amount of measurement error.”
BackgroundInfluentialEmmenegger, P., Schraff, D., & Walter, A. (2014) · VBN Forskningsportal (Aalborg Universitet) · Cited by 129
Ide, T. (2018) · Journal of Peace Research · doi:10.1177/0022343317750216 · Cited by 86
“In line with established standards (Hug, 2013; Skaaning, 2011), three kinds of robustness tests of the QCA results are conducted.”
BackgroundThiem, A. (2019) · Sociological Methods & Research · doi:10.1177/0049124119882463 · Cited by 76
“Other important exchanges include Cooper and Glaesser (2016), Schneider and Wagemann (2016), and Thiem (2016) on the enhanced standard analysis; Hug (2013), Thiem (2014), Hug (2014), and Thiem, Spöhel, and Duşa (2016) on sensitivity diagnostics; and Krogslund, Choi, and Poertner (2015), Rohlfing…”
BackgroundIde, T., Kristensen, A., & Bartusevičius, H. (2021) · Journal of Peace Research · doi:10.1177/0022343320966783 · Cited by 65
Critical Tension: Sufficiency and Parsimony in QCA
Duşa, A. (2019) · Sociological Methods & Research · doi:10.1177/0049124119882456 · Cited by 81
“B&T attempt a formal response to a series of critiques over the correctness of the QCA solutions (among others, Hug 2013; Lucas and Szatrowski 2014; Munck 2016; Seawright 2014; Tanner 2014), using a systematic method involving an a priori known causal structure and test whether the different QCA…”
MethodEnhancing Sensitivity Diagnostics for Qualitative Comparative Analysis: A Combinatorial Approach
Thiem, A., Spöhel, R., & Duşa, A. (2016) · Political Analysis · doi:10.1093/pan/mpv028 · Cited by 53
“…has been achieved in analyzing the effects of missing data and measurement error in other, more established methodological frameworks such as regression analysis (e.g., Wooldridge 2002, 70–81; Gelman and Hill 2007, 529–43), Hug (2013) presents the only serious attempt so far with respect to QCA.”
BackgroundMethodInfluentialCausality and History: Modes of Causal Investigation in Historical Social Sciences
Ermakoff, I. (2019) · Annual Review of Sociology · doi:10.1146/annurev-soc-073117-041140 · Cited by 64
Resisting Redevelopment: Protest in Aspiring Global Cities
Pasotti, E. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108775700 · Cited by 55
What did I leave out? Omitted variables in regression and qualitative comparative analysis
Radaelli, C. M., & Wagemann, C. (2018) · European Political Science · doi:10.1057/s41304-017-0142-7 · Cited by 48
A qualitative assessment of QCA: method stretching in large-N studies and temporality
(2022) · doi:10.1007/s11135-021-01278-5 · Cited by 35
“2009: 17), descriptive analyses, or to test deterministic hypotheses (Hug 2013: 252).”
BackgroundMethodInfluentialRobustness and Model Selection in Configurational Causal Modeling
Parkkinen, V., & Baumgärtner, M. (2021) · Sociological Methods & Research · doi:10.1177/0049124120986200 · Cited by 40
“But that finding, rather than yielding a meaningful estimate of the robustness of CCM models and demonstrating their unreliability, as Hug (2013) and Krogslund et al. (2015) submit, merely exhibits that a robustness measure rewarding invariance is at cross purposes with CCMs.”
BackgroundMethodThe Double Bind of Qualitative Comparative Analysis
Arel‐Bundock, V. (2019) · Sociological Methods & Research · doi:10.1177/0049124119882460 · Cited by 37
“Several researchers have used computer simulations to probe the robustness and accuracy of the algorithmic tools developed in the QCA tradition (e.g., Baumgartner and Ambu¨hl 2019; Hug 2013; Krogslund, Choi, and Poertner 2014; Lucas and Szatrowski 2014).”
MethodQualitative Comparative Analysis in Education Research: Its Current Status and Future Potential
Cilesiz, S., & Greckhamer, T. (2020) · Review of Research in Education · doi:10.3102/0091732x20907347 · Cited by 63
“29333/ejmste/83608 Hug, S. (2013). Qualitative comparative analysis: How inductive use and measurement error lead to problematic inference. Political Analysis, 21(2), 252–265. https://doi.org/10.1093/ pan/mps061 *Jain, V., & Jain, P. (2018). Affect vs cognition as antecedents of selection behaviour of elective courses…”
BackgroundInfluentialAggregation Bias and the Analysis of Necessary and Sufficient Conditions in fsQCA
Braumoeller, B. F. (2016) · Sociological Methods & Research · doi:10.1177/0049124116672701 · Cited by 30
Khizar, H. M. U., Iqbal, M. J., Murshed, F., & Ahsan, M. (2023) · Journal of Macromarketing · doi:10.1177/02761467231203311 · Cited by 41
“For instance, fuzzy-set scoring can result in measurement errors and misleading inferences (Hug 2013).”
BackgroundSet-Theoretic Comparative Methods: Less Distinctive Than Claimed
Paine, J. (2016) · Comparative Political Studies · doi:10.1177/0010414014564851 · Cited by 28
“Especially considering that many recent simulation studies demonstrate QCA algorithmic results are unreliable under various minor perturbations (Hug, 2013; Krogslund, Choi, & Poertner, 2014; Lucas & Szatrowski, 2014; Seawright, 2014), it will be valuable for STCM scholars to address how their…”
BackgroundNeo, R. (2021) · Political Studies Review · doi:10.1177/14789299211013984 · Cited by 35
“First, given that one of the main characteristics of QCA is that it can only treat dichotomized variables (present or not present), scholars have argued that this dichotomization can result in a loss of a great deal of information (Hug, 2013).”
BackgroundIngrams, A. R. (2017) · Public Administration · doi:10.1111/padm.12319 · Cited by 32
Collier, D. (2014) · Sociological Methodology · doi:10.1177/0081175014542568 · Cited by 30
de Andrade, L., Lynch, C. A., Carvalho, E., Rodrigues, C. G., Vissoci, J. R. N., Passos, G. F., Pietrobon, R. S., Nihei, O. K., & Carvalho, M. D. D. B. (2014) · PLoS ONE · doi:10.1371/journal.pone.0103577 · Cited by 31
“These PI state what combinations of conditions are necessary or sufficient for the outcome to occur or not [35,39,43].”
BackgroundQualitative comparative analysis for WASH research and practice
Kaminsky, J., & Jordan, E. A. (2017) · Journal of Water Sanitation and Hygiene for Development · doi:10.2166/washdev.2017.240 · Cited by 28
Co-location as a catalyst for service innovation: a study of Scottish health and social care
Memon, A., & Kinder, T. (2016) · Public Management Review · doi:10.1080/14719037.2016.1177107 · Cited by 28
Seawright, J. (2014) · Sociological Methodology · doi:10.1177/0081175014542082 · Cited by 25
“Hug (2013), for example, used simulations to show that measurement error can lead QCA to badly misrepresent the relationships in the data, and Lucas and Szatrowski (this volume, pp. 1–79) use similar techniques to demonstrate (among a wealth of other results) that findings of causal asymmetry in…”
MethodSet-theoretic methods in democratization research: an evaluation of their uses and contributions
Møller, J. Ø., & Skaaning, S. (2018) · Democratization · doi:10.1080/13510347.2018.1449208 · Cited by 28
Different algorithms, different models
Swiatczak, M. D. (2021) · Quality & Quantity · doi:10.1007/s11135-021-01193-9 · Cited by 29
“However, their use is still surprisingly scarce compared to the wide spread of CCM applications (for exceptions, see e.g. Arel-Bundock 2019; Baumgartner and Ambühl 2020a; Baumgartner and Thiem 2020; Cooper and Glaesser 2016a; Du ṣ a 2019b; Hug 2013; Krogslund et al. 2015).”
BackgroundBuche, J., & Siewert, M. Β. (2015) · Zeitschrift für Soziologie · doi:10.1515/zfsoz-2015-0602 · Cited by 32
“…die Diskussion siehe Lucas & Szatrowski 2014; Collier 2014; Ragin 2014; Seawright 2014; Vaisey 2014; Krogslund & Michel 2014; Krogslund et al. 2015; Hug 2013, 2014; Thiem 2014b; Schneider & Rohlfing 2014. theoretischem und fallorientiertem Ansatz zur Analyse von notwendigen und hinreichenden…”
BackgroundLogical Constraints: The Limitations of QCA in Social Science Research
Clarke, K. A. (2020) · Political Analysis · doi:10.1017/pan.2020.7 · Cited by 20
Dörfler, T., Holzinger, K., & Biesenbender, J. (2017) · International Journal of Public Administration · doi:10.1080/01900692.2017.1295267 · Cited by 20
“For example, critics point to lacking robustness of results, or they emphasize the advantages of purely quantitative methods over QCA (e.g., Hug, 2013; Lucas & Szatrowski, 2014; Skaaning, 2011).”
BackgroundWhat Is Our Purpose Here? Network Relationships and Goal Congruence in a Goal-Directed Network
Lemaire, R. H. (2019) · The American Review of Public Administration · doi:10.1177/0275074019879506 · Cited by 24
“There are a number of issues with QCA, including issues due to limited diversity (Seawright, 2014), sensitivity to measurement error (Hug, 2013) and model and parameter specification (Krogslund, Choi, & Poertner, 2015), and the occurrence of false positives due to results produced by chance…”
BackgroundRohlfing, I. (2018) · Political Analysis · doi:10.1017/pan.2017.30 · Cited by 18
“9 See (Ragin 2008, ch. 7) on the threshold in a truth table analysis.”
BackgroundThiem, A. (2014) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.894403 · Cited by 18
A general framework for learning about research designs †Preprint
Blair, G., Cooper, J., Coppock, A., & Humphreys, M. (2018) · doi:10.31222/osf.io/m6ws3
Modeling Causal Irrelevance in Evaluations of Configurational Comparative Methods
Thiem, A., & Baumgärtner, M. (2016) · Sociological Methodology · doi:10.1177/0081175016654736 · Cited by 18
Qualitative Comparative Analysis
Siewert, M. Β. (2017) · Springer eBooks · doi:10.1007/978-3-658-17189-6_10 · Cited by 15
Qualitative Comparative Analysis
Wagemann, C. (2015) · Springer eBooks · doi:10.1007/978-3-658-01968-6_17 · Cited by 16
Social causation and protest mobilization: why temporality and interaction matter
Anisin, A. (2017) · Territory Politics Governance · doi:10.1080/21622671.2017.1306455 · Cited by 14
“Along these lines, Hug (2013) noted that many strands of social science are not yet well-developed enough to test deterministic hypotheses.”
BackgroundYan, H., van der Heijden, J., & van Rooij, B. (2015) · Regulation & Governance · doi:10.1111/rego.12095 · Cited by 29
“It goes without saying that QCA is not without criticism, and we reflect on problems we have experienced in the concluding section (see also, Emmenegger 2012; Hug 2013; Rihoux & Marx 2013).”
BackgroundMethodHow Do They Get In? Radical Parties and Government Participation in European Democracies
Fagerholm, A. (2019) · Government and Opposition · doi:10.1017/gov.2019.24 · Cited by 11
“…standards of good practice (Schneider andWagemann 2010, 2012; Wagemann and Schneider 2015) and, when appropriate, suggestions made in more recent evaluations of the stability of CCMs (Baumgartner and Thiem 2017a, 2017b; cf. also Hug 2013; Krogslund et al. 2015; Lucas and Szatrowski 2014).”
BackgroundCase-to-factor Ratios and Model Specification in Qualitative Comparative Analysis
Thiem, A., & Mkrtchyan, L. (2023) · Field Methods · doi:10.1177/1525822x231159458 · Cited by 14
Unifying the Study of Asymmetric Hypotheses
Rosenberg, A. S., Knuppe, A. J., & Braumoeller, B. F. (2017) · Political Analysis · doi:10.1017/pan.2017.16 · Cited by 11
“Hug (2013) shows that QCA can produce misleading inferences if measurement error is not taken into account.”
BackgroundMethodInfluentialIntroduction: absorbing the four methodological disruptions in democratization research?
Coppedge, M., & Kuehn, D. (2018) · Democratization · doi:10.1080/13510347.2018.1509851 · Cited by 12
25 années de QCA ( Qualitative Comparative Analysis ) : quel chemin parcouru ?
Rihoux, B., Marx, A., & Álamos‐Concha, P. (2015) · Revue internationale de politique comparée · doi:10.3917/ripc.212.0061 · Cited by 20
Bayesian Rule Set: A Quantitative Alternative to Qualitative Comparative Analysis
Chiu, A., & Xu, Y. (2020) · The Journal of Politics · doi:10.1086/720791 · Cited by 12
A Larger-N, Fewer Variables Problem? The Counterintuitive Sensitivity Of Qca
Krogslund, C. C., & Michel, K. (2014) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.894604 · Cited by 8
Batho, A., Kneale, D., Sutcliffe, K., & Williams, A. C. D. C. (2021) · Pain · doi:10.1097/j.pain.0000000000002242 · Cited by 9
Sampling Theoretically for Comparison
Adebayo, A. O., & Ackers, B. (2021) · The Electronic Journal of Business Research Methods · doi:10.34190/ejbrm.19.1.2434 · Cited by 20
Fahy, L., Lee, L. S., Newlove, L., Wood, L., & O’Driscoll, C. (2025) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12576 · Cited by 9
“While QCA has limitations, including arbitrary set inclusion criteria in crisp-set analysis (Vink & Vliet, 2009) and assumptions of error-free measurement (Hug, 2013), it moves beyond theme identification to examine how different factor combinations influence therapy engagement.”
Leonidou, E., Battisti, E., Ahmad, M. I., & Campo, R. (2024) · International Marketing Review · doi:10.1108/imr-04-2022-0103 · Cited by 7
Zhang, Y., Wang, X., Wan, S., et al. (2023) · Sustainability · doi:10.3390/su15086743 · Cited by 6
Fiscal Centralization in Times of Crisis: Evidence from 11 Federal StatesPreprint
Trein, P., & Ruiz-Palmero, C. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2548185 · Cited by 4
Addressing the challenges in using qualitative data in qualitative comparative analysis
de Block, D., & Vis, B. (2017) · Data Archiving and Networked Services (DANS) · Cited by 6
Qualitative Comparative Analysis (QCA)
Marx, A. (2016) · Oxford University Press eBooks · doi:10.1093/obo/9780199756384-0188 · Cited by 6
Yörük, E., & Gençer, A. Ş. (2022) · Journal of Comparative Policy Analysis Research and Practice · doi:10.1080/13876988.2021.2004541 · Cited by 5
“Furthermore, a typological theory requires fewer assumptions in making inferences than in other methods such as Qualitative Comparative Analysis (QCA) (Hug 2013).”
MethodStolz, E. (2015) · International Journal of Sociology · doi:10.1080/00207659.2015.1098177 · Cited by 5
“When we say “in principle,” this is because the modeling of both fixed and random effects in MLRMs relies on assumptions of normality and large sample sizes, whereby the number of countries constitutes the main bottleneck (Hox 2010).”
BackgroundUma revisão sobre a pesquisa qualitativa em ciências sociais aplicadas
Brizolla, M. M. B., Petry, J. F., Uchôa, A. G. F., & Ferreira, H. L. B. (2020) · UFAM Business Review - UFAMBR · doi:10.47357/ufambr.v2i3.8087 · Cited by 22
Transcending the National: QCA Insights into Multilevel Party Organization in the EU
Pittoors, G., & Vandeleene, A. (2025) · Government and Opposition · doi:10.1017/gov.2025.10009 · Cited by 4
“One should also be aware of the many criticisms made of QCA, particularly regarding the sweeping effect of measurement error in the QCA algorithms (Baumgartner and Thiem 2017; Collier 2014; Hug 2013).”
Validation strategies in historical sociology (and beyond)
Ermakoff, I. (2022) · Edward Elgar Publishing eBooks · doi:10.4337/9781789909432.00019 · Cited by 5
Fujita, T. (2023) · Japanese Journal of Administrative Science · doi:10.5651/jaas.34.75 · Cited by 7
“…難である。そこでは,独立変数(原因)の数値 の組合せすべてについて,従属変数(結果)と の対応関係が検討されるわけではない。諸原因 の特定の組合せばかりが現実に存在する場合, その推定はあやしいものであるにもかかわら ず,そのことが顧みられることはない。複数の 原因の組合せの効果を計量分析で確認する方法 として,たとえば交互作用項を投入することも できる。だが,たとえば 3つ以上の原因からな る結合因果を分析しようとすれば,多重共線性 の問題や必要な事例数の問題が生じるため,現 実的ではない。 したがって,多元結合因果という多様で複雑 な因果を捉える上で,少数事例研究による事例…”
The logico-formalist turn in comparative and case study methods: a critical realist critique
Pula, B. (2020) · International Journal of Social Research Methodology · doi:10.1080/13645579.2020.1823063 · Cited by 5
“While critical realism also acknowledges these problems, Ragin’s solution treats them not as consequences of the theoretical and practical limits of research, but as equivalent to stochastic problems of ‘random variation’ and ‘measurement error’ (Hug, 2013; King et al., 1994, p. 59; Paine, 2016).”
BackgroundIntroduction to the Virtual Issue
Alvarez, R. M., & Katz, J. N. (2014) · Political Analysis · doi:10.1017/s1047198700014352 · Cited by 4
Morse, Y. L. (2020) · Democratization · doi:10.1080/13510347.2020.1803283 · Cited by 5
Collier, D. (2014) · SSRN Electronic Journal · Cited by 5
Advocacy Coalitions ‘In Power’ and Major Policy Change in Canada: A Fuzzy Set Investigation
Heinmiller, B. T. (2025) · Springer eBooks · doi:10.1007/978-3-031-85554-2_4 · Cited by 3
Outcomes of Comprehensive Service Delivery and Early Interventions in Countering Youth Exclusion
Thøgersen, M. (2024) · Journal of Applied Youth Studies · doi:10.1007/s43151-024-00126-1 · Cited by 6
“Furthermore, despite the increasing use of QCA in exploring causal relationships in social science, there are critical perspectives regarding its analytical process and suitability compared to other methods (e.g. Tanner 2014; Paine 2016; Lucas and Szatrowski 2014; Hug 2013).”
Problematic Tools: Introduction to Symposium on Set Theory in Social SciencePreprint
Collier, D. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2754821 · Cited by 3
Calibrating and Combining Sets
Oana, I., Schneider, C. Q., & Thomann, E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781009006781.007 · Cited by 3
Quantitative Methoden in den Internationalen Beziehungen
Ruhe, C., Schneider, G., & Spilker, G. (2022) · Springer Reference Sozialwissenschaften · doi:10.1007/978-3-531-19954-2_25-2 · Cited by 2
Educational Support Surrounding Chronic Kidney Disease: Qualitative Enquiry
Muhammad, S. N., Orzechowska, K., Gardner, J., Gardner, V., Pizon, K., Carson, A., & Christine, H. (2023) · Turkish Journal of Nephrology · doi:10.5152/turkjnephrol.2023.22100394 · Cited by 2
“30”
BackgroundMethodIntroduction: QCA in a Nutshell
Oana, I., Schneider, C. Q., & Thomann, E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781009006781.005 · Cited by 2
Ide, T. (2015) · MPG.PuRe (Max Planck Society) · Cited by 4
The Quality of Representation in Latin America: Linking Citizens with Political Parties
Ruth, S. P. (2014) · Kölner Universitäts PublikationsServer (Universität zu Köln) · Cited by 2
Explaining Asylum Law Using Qualitative Comparative Analysis
Kretsedemas, P. (2024) · Laws · doi:10.3390/laws13040053 · Cited by 2
“This premise has been used to raise questions about the validity of QCA research that takes an inductive approach to hypothesis testing (Hug 2013).”
Lobbying coalitions and their determinants: a systematic literature review
Pritoni, A., Bitonti, A., & Montalbano, G. (2025) · European Political Science Review · doi:10.1017/s1755773925100143 · Cited by 3
“…to establish itself in the study of interest groups, partly because of some limitations related to aggregation biases, problematic inference (Hug, 2013) and the impossibility to allow for counterfactuals (Clarke, 2020), the conception of conjunctural causality that characterizes it…”
Response to Greckhamer et al (2018): Studying configurations with qualitative comparative analysis
Miller, D. L. (2018) · Strategic Organization · doi:10.1177/1476127018804528 · Cited by 1
“Of course, like clustering, QCA has its own subjectivity and reliability challenges, including those relating to inappropriate causal inference (see the detailed critiques and commentaries by Collier (2014), Hug (2013), Lucas and Szatrowski (2014), Ragin (2014) and Seawright (2014).”
BackgroundSchmiedl, M., & Prouza, J. (2021) · Central European Journal of International and Security Studies · doi:10.51870/cejiss.a150202 · Cited by 1
Pasotti, E. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108775700.012 · Cited by 1
Quantitative Methoden in den Internationalen Beziehungen
Ruhe, C., Schneider, G., & Spilker, G. (2014) · VS Verlag für Sozialwissenschaften eBooks · doi:10.1007/978-3-531-19954-2_25-1 · Cited by 1
Identifying Student Motivation in Learning Islamic Education in Senior High School
Saputri, T. E., Izmi, N., Putera, O., & Irayanti, I. (2022) · Ahlussunnah Journal of Islamic Education · doi:10.58485/jie.v1i1.114 · Cited by 4
“Qualitative research is research used to examine the state of natural objects, namely something that is as it is, not conditioned by circumstances and conditions (Hasanah, 2021; Hug, 2013).”
Metadata
- Authors
- Hug, S.
- Affiliations
- University of Geneva (CH)
- Corresponding author
- Hug, Simon
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2013
- Volume
- 21
- Issue
- 2
- Pages
- 252-265
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W2142575249
- Cited by
- 137
- References
- 83
- Citations by year
- 2025: 6 · 2024: 4 · 2023: 7 · 2022: 6 · 2021: 21 · 2020: 25 · 2019: 7 · 2018: 12 · 2017: 12 · 2016: 10 · 2015: 15 · 2014: 12
- FWCI
- 36.64
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Political Analysis (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Archive ouverte UNIGE (University of Geneva) (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Qualitative Comparative Analysis Research
- Social Sciences › Sociology and Political Science › Political Conflict and Governance
- Social Sciences › Business, Management and Accounting › Strategy and Management › Political Influence and Corporate Strategies
- Keywords
- qualitative comparative analysis, Boolean algebra, measurement error, inductive reasoning, exploratory analysis, Monte Carlo simulation, causal inference, political science methodology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.