Comparing and Combining List and Endorsement Experiments: Evidence from Afghanistan
The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
List and endorsement experiments are becoming increasingly popular among social scientists as indirect survey techniques for sensitive questions. When studying issues such as racial prejudice and support for militant groups, these survey methodologies may improve the validity of measurements by reducing nonresponse and social desirability biases. We develop a statistical test and multivariate regression models for comparing and combining the results from list and endorsement experiments. We demonstrate that when carefully designed and analyzed, the two survey experiments can produce substantively similar empirical findings. Such agreement is shown to be possible even when these experiments are applied to one of the most challenging research environments: contemporary Afghanistan. We find that both experiments uncover similar patterns of support for the International Security Assistance Force (ISAF) among Pashtun respondents. Our findings suggest that multiple measurement strategies can enhance the credibility of empirical conclusions. Open‐source software is available for implementing the proposed methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Inference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627 · cited 1×
“We run three chains with overdispersed starting values to monitor convergence (Gelman and Rubin, 1992).”
MethodThe Logic of Violence in Civil War
Kalyvas, S. N. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511818462 · Cited by 3,314
Randomized Response: A Survey Technique for Eliminating Evasive Answer Bias
Warner, S. L. (1965) · Journal of the American Statistical Association · doi:10.1080/01621459.1965.10480775 · Cited by 3,255 · cited 1×
“While 1Another popular survey methodology is randomized response technique (Warner, 1965), which is not studied here (see also Gingerich, 2010). they may reduce non-response and social desirability biases, indirect questioning techniques are often susceptible to measurement error and are affected…”
BackgroundThe Logic of Violence in Civil War
(2006) · The Logic of Violence in Civil War · doi:10.1017/cbo9780511818462.001 · Cited by 2,402
FISHER, R. A. (1915) · Biometrika · doi:10.1093/biomet/10.4.507 · Cited by 1,574 · cited 1×
“Typically, this is done using the two-sample z-test after applying Fisher’s Z transformation to each of the two sample correlations (Fisher, 1915).”
MethodCan Hearts and Minds Be Bought? The Economics of Counterinsurgency in Iraq
Berman, E., Shapiro, J. N., & Felter, J. H. (2011) · Journal of Political Economy · doi:10.1086/661983 · Cited by 694 · cited 1×
“…war: (1) the amount of aid or development funds allocated to a given village or area (Beath, Christia and Enikolopov, 2011; Berman, Shapiro and Felter, 2011; U.S. Army, 2007), here measured by district-level Commander’s Emergency Response Program, or CERP, spending (top panel) and (2) the…”
Statistical Analysis of List Experiments
Blair, G., & Imai, K. (2012) · Political Analysis · doi:10.1093/pan/mpr048 · Cited by 639
Self-citationThe Logic of the Survey Experiment Reexamined
Gaines, B. J., Kuklinski, J. H., & Quirk, P. J. (2007) · Political Analysis · doi:10.1093/pan/mpl008 · Cited by 553
Social desirability bias in voter turnout reports: Tests using the item count technique
Holbrook, A. L., & Krosnick, J. A. (2010) · Public Opinion Quarterly · doi:10.1093/poq/nfp065 · Cited by 539 · cited 1×
“…popular as survey methodology to overcome this measurement problem (see for example references in Tsuchiya, Hirai and Ono, 2007; Corstange, 2009; Holbrook and Krosnick, 2010; Glynn, 2010; Bullock, Imai and Shapiro, 2011; Gonzalez-Ocantos et al., 2012; Blair and Imai, 2012).1 These survey…”
MethodExplaining Support for Combatants during Wartime: A Survey Experiment in Afghanistan
LYALL, J., BLAIR, G., & IMAI, K. (2013) · American Political Science Review · doi:10.1017/s0003055413000403 · Cited by 527
Self-citationRacial Attitudes and the "New South"
Kuklinski, J. H., Cobb, M. D., & Gilens, M. (1997) · The Journal of Politics · doi:10.1017/s0022381600053470 · Cited by 519
Vote Buying and Social Desirability Bias: Experimental Evidence from Nicaragua
Gonzalez‐Ocantos, E., de Jonge, C. K., Meléndez, C., Osorio, J., & Nickerson, D. W. (2012) · American Journal of Political Science · doi:10.1111/j.1540-5907.2011.00540.x · Cited by 498 · cited 2×
“ogy to overcome this measurement problem (see, e.g., Blair and Imai 2012; Bullock, Imai, and Shapiro 2011; Corstange 2009; Glynn 2013; Gonzalez-Ocantos et al. 2012; Holbrook and Krosnick 2010; Tsuchiya, Hirai, and Ono 2007).”
What Can We Learn with Statistical Truth Serum?
Glynn, A. N. (2013) · Public Opinion Quarterly · doi:10.1093/poq/nfs070 · Cited by 411
Multivariate Regression Analysis for the Item Count Technique
Imai, K. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.ap10415 · Cited by 323 · cited 4×
“New statistical methods have been developed so that researchers can conduct multivariate regression analysis for each of these survey techniques (Corstange, 2009; Glynn, 2010; Bullock, Imai and Shapiro, 2011; Imai, 2011; Blair and Imai, 2012).”
BackgroundMethodInfluentialSelf-citationPoverty and Support for Militant Politics: Evidence from Pakistan
Blair, G., Christine Fair, C., Malhotra, N., & Shapiro, J. N. (2013) · American Journal of Political Science · doi:10.1111/j.1540-5907.2012.00604.x · Cited by 251
Self-citationSensitive Questions, Truthful Answers? Modeling the List Experiment with LISTIT
Corstange, D. (2009) · Political Analysis · doi:10.1093/pan/mpn013 · Cited by 248
Accelerating Monte Carlo Markov chain convergence for cumulative-link generalized linear models
Cowles, M. K. (1996) · Statistics and Computing · doi:10.1007/bf00162520 · Cited by 224
Winning Hearts and Minds through Development? Evidence from a Field Experiment in Afghanistan
Beath, A., Christia, F., & Сергеевич, Е. Р. (2012) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-6129 · Cited by 196
Statistical Analysis of Endorsement Experiments: Measuring Support for Militant Groups in Pakistan
Bullock, W., Imai, K., & Shapiro, J. N. (2011) · Political Analysis · doi:10.1093/pan/mpr031 · Cited by 158
Self-citationBlock Total Response as an Alternative to the Randomized Response Method in Surveys
Raghavarao, D., & Federer, W. T. (1979) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1979.tb01055.x · Cited by 145 · cited 1×
“List experiments, also known as the item count technique, use aggregation: respondents are asked to count the number of items on a list which includes a sensitive item (Raghavarao and Federer, 1979; Miller, 1984; Kuklinski, Cobb and Gilens, 1997).”
MethodA Study of the Properties of the Item Count Technique
Tsuchiya, T., Hirai, Y., & Ono, S. (2007) · Public Opinion Quarterly · doi:10.1093/poq/nfm012 · Cited by 136
Rebellion and Authority: An Analytic Essay on Insurgen Conflicts
Leites, N., & Wolf, C. (1970) · RAND Corporation eBooks · Cited by 126
Gingerich, D. W. (2010) · Political Analysis · doi:10.1093/pan/mpq010 · Cited by 98 · cited 1×
“While 1Another popular survey methodology is randomized response technique (Warner, 1965), which is not studied here (see also Gingerich, 2010). they may reduce non-response and social desirability biases, indirect questioning techniques are often susceptible to measurement error and are affected…”
BackgroundTreatment Spillover Effects across Survey Experiments
Transue, J. E., Lee, D. J., & Aldrich, J. H. (2009) · Political Analysis · doi:10.1093/pan/mpn012 · Cited by 58 · cited 1×
“Specifically, we assume that the latent support levels in both experiments have the identical distribution by imposing the restriction γ = λ/ω, which directly connects the two models.”
Backgroundlist: Statistical Methods for the Item Count Technique and List Experiment
Blair, G., Chou, W., & Imai, K. (2010) · doi:10.32614/cran.package.list · Cited by 54
Self-citationGodlee, F. (2005) · BMJ · doi:10.1136/bmj.330.7502.0-g · Cited by 46
1 Winning Hearts and Minds? Evidence from a Field Experiment in Afghanistan1
Beath, A., Christia, F., & Сергеевич, Е. Р. (n.d.) · Cited by 15
Cited by
Frye, T., Gehlbach, S., Marquardt, K. L., & Reuter, O. J. (2017) · Post-Soviet Affairs · doi:10.1080/1060586x.2016.1144334 · Cited by 297
BLAIR, G., COPPOCK, A., & MOOR, M. (2020) · American Political Science Review · doi:10.1017/s0003055420000374 · Cited by 290
Self-citationCoffman, K. B., Coffman, L. C., & Ericson, K. M. M. (2017) · Management Science · doi:10.1287/mnsc.2016.2503 · Cited by 287
Method Matters: The Underreporting of Intimate Partner Violence
Cullen, C. (2023) · The World Bank Economic Review · doi:10.1093/wber/lhac022 · Cited by 116
Drought, Local Institutional Contexts, and Support for Violence in Kenya
Linke, A. M., Witmer, F. D. W., O’Loughlin, J., McCabe, J. T., & Tir, J. (2018) · Journal of Conflict Resolution · doi:10.1177/0022002717698018 · Cited by 77
Metadata
- Authors
- Blair, G., Imai, K., & Lyall, J.
- Venue
- American Journal of Political Science
- Publisher
- Wiley
- Published
- 2014-02-12
- Volume
- 58
- Issue
- 4
- Pages
- 1043-1063
- ISSN
- 0092-5853 (ISSN-L)
- 1540-5907
- OpenAlex
- W2097414408
- Cited by
- 187
- References
- 27
- FWCI
- 15.47
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.