Abstraction and Detail in Experimental Design
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Political scientists designing experiments often face the question of how abstract or detailed their experimental stimuli should be. Typically, this question is framed in terms of trade‐offs relating to experimental control and generalizability: the more context introduced into studies, the less control, and the more difficulty generalizing the results. Yet, we have reason to question this trade‐off, and there is relatively little systematic evidence to rely on when calibrating the degree of abstraction in studies. We make two contributions. First, we provide a theoretical framework that identifies and considers the consequences of three dimensions of abstraction in experimental design: situational hypotheticality, actor identity, and contextual detail. Second, we replicate and extend three survey experiments, varying these levels of abstraction. We find no evidence that situational hypotheticality substantively changes results in any of our studies, but do find that increased contextual detail dampens treatment effects, and that the salience of actor identities moderates results in our endorsement experiment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Concept Misformation in Comparative Politics
Sartori, G. (1970) · American Political Science Review · doi:10.2307/1958356 · Cited by 3,318 · cited 1×
“For our purposes, we borrow from construal level theory in defining abstraction as a higher-level representation (Sartori, 1970, 1040-46; Trope and Liberman, 2003).”
MethodTrope, Y., & Liberman, N. (2003) · Psychological Review · doi:10.1037/0033-295x.110.3.403 · Cited by 3,006
How Elite Partisan Polarization Affects Public Opinion Formation
DRUCKMAN, J. N., PETERSON, E., & SLOTHUUS, R. (2013) · American Political Science Review · doi:10.1017/s0003055412000500 · Cited by 1,569
What Triggers Public Opposition to Immigration? Anxiety, Group Cues, and Immigration Threat
Brader, T., Valentino, N. A., & Suhay, E. (2008) · American Journal of Political Science · doi:10.1111/j.1540-5907.2008.00353.x · Cited by 1,313
The Use of Vignettes in Survey Research
Alexander, C. S., & Becker, H. J. (1978) · Public Opinion Quarterly · doi:10.1086/268432 · Cited by 1,169 · cited 1×
“Rich and detailed stimuli are in some ways a response to the “major problem in public opinion and survey research”: the “ambiguity that often arises when survey respondents are asked to make decisions and judgments from rather abstract and limited information” (Alexander and Becker, 1978, 103).”
BackgroundThe Logic of the Survey Experiment Reexamined
Gaines, B. J., Kuklinski, J. H., & Quirk, P. J. (2007) · Political Analysis · doi:10.1093/pan/mpl008 · Cited by 553
Demand Effects in Survey Experiments: An Empirical Assessment
MUMMOLO, J., & PETERSON, E. (2019) · American Political Science Review · doi:10.1017/s0003055418000837 · Cited by 548
Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach
Coppock, A. (2019) · Political Science Research and Methods · doi:10.1017/psrm.2018.10 · Cited by 515
Are Survey Experiments Externally Valid?
BARABAS, J., & JERIT, J. (2010) · American Political Science Review · doi:10.1017/s0003055410000092 · Cited by 481
Nicholson, S. P. (2012) · American Journal of Political Science · doi:10.1111/j.1540-5907.2011.00541.x · Cited by 455
Information equivalence in survey experiments
Dafoe, A., Zhang, B., & Caughey, D. (2018) · Political Analysis · doi:10.1017/pan.2018.9 · Cited by 356
How Quickly We Forget: The Duration of Persuasion Effects From Mass Communication
Hill, S. J., Lo, J., Vavreck, L., & Zaller, J. (2013) · Political Communication · doi:10.1080/10584609.2013.828143 · Cited by 238 · cited 1×
“The first is a standard “online processing model” (Hill et al., 2013) in which respondents keep a running tally of evaluations that are updated when they come into contact with new information.”
Background
Cited by
The Liar’s Dividend: Can Politicians Claim Misinformation to Evade Accountability?
Schiff, K. J., Schiff, D. S., & Bueno, N. (2023) · doi:10.31235/osf.io/x43ph · Cited by 8
The Liar’s Dividend: Can Politicians Use Deepfakes and Fake News to Evade Accountability?
Schiff, K. J., Schiff, D. S., & Bueno, N. (2022) · doi:10.31235/osf.io/q6mwn · Cited by 6
Metadata
- Authors
- Brutger, R., Kertzer, J. D., Renshon, J., Tingley, D., & Weiss, C. M.
- Venue
- American Journal of Political Science
- Publisher
- Wiley
- Published
- 2022-05-31
- Volume
- 67
- Issue
- 4
- Pages
- 979-995
- ISSN
- 0092-5853 (ISSN-L)
- 1540-5907
- OpenAlex
- W4281734124
- Cited by
- 163
- References
- 12
- Type
- conference abstract
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.