The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Measuring Subgroup Preferences in Conjoint Experiments
Open accessAbstract
Conjoint analysis is a common tool for studying political preferences. The method disentangles patterns in respondents’ favorability toward complex, multidimensional objects, such as candidates or policies. Most conjoints rely upon a fully randomized design to generate average marginal component effects (AMCEs). They measure the degree to which a given value of a conjoint profile feature increases, or decreases, respondents’ support for the overall profile relative to a baseline, averaging across all respondents and other features. While the AMCE has a clear causal interpretation (about the effect of features), most published conjoint analyses also use AMCEs to describe levels of favorability. This often means comparing AMCEs among respondent subgroups. We show that using conditional AMCEs to describe the degree of subgroup agreement can be misleading as regression interactions are sensitive to the reference category used in the analysis. This leads to inferences about subgroup differences in preferences that have arbitrary sign, size, and significance. We demonstrate the problem using examples drawn from published articles and provide suggestions for improved reporting and interpretation using marginal means and an omnibus F-test. Given the accelerating use of these designs in political science, we offer advice for best practice in analysis and presentation of results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Shmueli, G. (2010) · Statistical Science · doi:10.1214/10-sts330 · Cited by 2,517
Hainmueller, J., Hopkins, D. J., & Yamamoto, T. (2014) · Political Analysis · doi:10.1093/pan/mpt024 · Cited by 1,990
The Hidden American Immigration Consensus: A Conjoint Analysis of Attitudes toward Immigrants
Hainmueller, J., & Hopkins, D. J. (2015) · American Journal of Political Science · doi:10.1111/ajps.12138 · Cited by 952
How economic, humanitarian, and religious concerns shape European attitudes toward asylum seekers
Bansak, K., Hainmueller, J., & Hangartner, D. (2016) · Science · doi:10.1126/science.aag2147 · Cited by 811
Dynamic Documents with R and knitr
Xie, Y. (2016) · Chapman & Hall/CRC eBooks · doi:10.1201/b15166 · Cited by 746
Population-Based Survey Experiments
Mutz, D. C. (2011) · doi:10.1515/9781400840489 · Cited by 674
Cambridge Handbook of Experimental Political Science
(2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511921452 · Cited by 611
The Logic of the Survey Experiment Reexamined
Gaines, B. J., Kuklinski, J. H., & Quirk, P. J. (2007) · Political Analysis · doi:10.1093/pan/mpl008 · Cited by 553
The Ties That Double Bind: Social Roles and Women's Underrepresentation in Politics
TEELE, D. L., KALLA, J., & ROSENBLUTH, F. (2018) · American Political Science Review · doi:10.1017/s0003055418000217 · Cited by 382
The Growth and Development of Experimental Research in Political Science
DRUCKMAN, J. N., GREEN, D. P., KUKLINSKI, J. H., & LUPIA, A. (2006) · American Political Science Review · doi:10.1017/s0003055406062514 · Cited by 374
Mass support for global climate agreements depends on institutional design
Bechtel, M. M., & Scheve, K. F. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1306374110 · Cited by 350
Why Partisans Do Not Sort: The Constraints on Political Segregation
Mummolo, J., & Nall, C. (2017) · The Journal of Politics · doi:10.1086/687569 · Cited by 269
Ethnic Voting and Accountability in Africa: A Choice Experiment in Uganda
Carlson, E. (2015) · World Politics · doi:10.1017/s0043887115000015 · Cited by 266
Green, D. P., & Kern, H. L. (2012) · Public Opinion Quarterly · doi:10.1093/poq/nfs036 · Cited by 250
When Do Renters Behave Like Homeowners? High Rent, Price Anxiety, and NIMBYism
HANKINSON, M. (2018) · American Political Science Review · doi:10.1017/s0003055418000035 · Cited by 225
CARNES, N., & LUPU, N. (2016) · American Political Science Review · doi:10.1017/s0003055416000551 · Cited by 223
Candidate Choice Without Party Labels: New Insights from Conjoint Survey Experiments
Kirkland, P. A., & Coppock, A. (2018) · Political Behavior · doi:10.1007/s11109-017-9414-8 · Cited by 214
The Structure of American Income Tax Policy Preferences
Ballard-Rosa, C., Martin, L., & Scheve, K. (2017) · The Journal of Politics · doi:10.1086/687324 · Cited by 187
Wright, M., Levy, M., & Citrin, J. (2016) · Political Behavior · doi:10.1007/s11109-015-9311-y · Cited by 180
Bechtel, M. M., Genovese, F., & Scheve, K. F. (2019) · British Journal of Political Science · doi:10.1017/s0007123417000205 · Cited by 166
Oliveros, V., & Schuster, C. (2017) · Comparative Political Studies · doi:10.1177/0010414017710268 · Cited by 151
News from the Other Side: How Topic Relevance Limits the Prevalence of Partisan Selective Exposure
Mummolo, J. (2016) · The Journal of Politics · doi:10.1086/685584 · Cited by 141
Grimmer, J., Messing, S., & Westwood, S. J. (2017) · Political Analysis · doi:10.1017/pan.2017.15 · Cited by 127
Sobolewska, M., Galandini, S., & Lessard-Phillips, L. (2016) · Journal of Ethnic and Migration Studies · doi:10.1080/1369183x.2016.1248377 · Cited by 120
Multi-dimensional preferences for labour market reforms: a conjoint experiment
Gallego, A., & Marx, P. (2016) · Journal of European Public Policy · doi:10.1080/13501763.2016.1170191 · Cited by 119
Exposure to Immigration and Admission Preferences: Evidence From France
Clayton, K., Ferwerda, J., & Horiuchi, Y. (2018) · Political Behavior · doi:10.1007/s11109-019-09550-z · Cited by 119
Causal Interaction in Factorial Experiments: Application to Conjoint Analysis
Egami, N., & Imai, K. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2018.1476246 · Cited by 114
Corruption, Accountability, and Gender: Do Female Politicians Face Higher Standards in Public Life?
Eggers, A. C., Vivyan, N., & Wagner, M. (2017) · The Journal of Politics · doi:10.1086/694649 · Cited by 113
How Political Signals Affect Public Support for Judicial Nominations
Sen, M. (2017) · Political Research Quarterly · doi:10.1177/1065912917695229 · Cited by 109
Policy design and domestic support for international bailouts
Bechtel, M. M., Hainmueller, J., & Margalit, Y. M. (2017) · European Journal of Political Research · doi:10.1111/1475-6765.12210 · Cited by 107
Population-Based Survey Experiments
Land, K. C. (2012) · Contemporary Sociology A Journal of Reviews · doi:10.1177/0094306112462561s · Cited by 103
Franchino, F., & Zucchini, F. (2014) · Political Science Research and Methods · doi:10.1017/psrm.2014.24 · Cited by 102
Cross-National Yardstick Comparisons: A Choice Experiment on a Forgotten Voter Heuristic
Hansen, K. M., Olsen, A. L., & Bech, M. (2014) · Political Behavior · doi:10.1007/s11109-014-9288-y · Cited by 85
Legislator Dissent as a Valence Signal
Campbell, R., Cowley, P., Vivyan, N., & Wagner, M. (2016) · British Journal of Political Science · doi:10.1017/s0007123416000223 · Cited by 78
House or home? Constituent preferences over legislator effort allocation
Vivyan, N., & Wagner, M. (2015) · European Journal of Political Research · doi:10.1111/1475-6765.12119 · Cited by 63
Sparse Estimation and Uncertainty with Application to Subgroup Analysis
Ratkovic, M., & Tingley, D. H. (2017) · Political Analysis · doi:10.1017/pan.2016.14 · Cited by 51
Carey, J., Carman, K., Clayton, K., Horiuchi, Y., Htun, M., & Ortiz, B. (2018) · Politics, Groups, and Identities · doi:10.1080/21565503.2018.1491866 · Cited by 37
cregg: Simple Conjoint Tidying, Analysis, and Visualization
Leeper, T. J. (2018) · doi:10.32614/cran.package.cregg · Cited by 18
Self-citation
Cited by
GRAHAM, M. H., & SVOLIK, M. W. (2020) · American Political Science Review · doi:10.1017/s0003055420000052 · Cited by 1,044
Factors Associated With US Adults’ Likelihood of Accepting COVID-19 Vaccination
Kreps, S., Prasad, S., Brownstein, J. S., Hswen, Y., Garibaldi, B. T., Zhang, B., & Kriner, D. L. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.25594 · Cited by 732
Divided by the vote: Affective polarization in the wake of the Brexit referendum
Hobolt, S. B., Leeper, T. J., & Tilley, J. (2021) · British Journal of Political Science · doi:10.1017/s0007123420000125 · Cited by 496
Self-citationResolving content moderation dilemmas between free speech and harmful misinformation
Kozyreva, A., Herzog, S. M., Lewandowsky, S., Hertwig, R., Lorenz-Spreen, P., Leiser, M., & Reifler, J. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2210666120 · Cited by 166
The partisan nature of support for democratic backsliding: A comparative perspective
Gidengil, E., Stolle, D., & Bergeron‐Boutin, O. (2022) · European Journal of Political Research · doi:10.1111/1475-6765.12502 · Cited by 162
Metadata
- Authors
- Leeper, T. J., Hobolt, S. B., & Tilley, J.
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2019-08-07
- Volume
- 28
- Issue
- 2
- Pages
- 207-221
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W2948893001
- Cited by
- 846
- References
- 38
- FWCI
- 237.55
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.