Does Survey Mode Still Matter? Findings from a 2010 Multi-Mode Comparison
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
In this article, we present data from a three-mode survey comparison study carried out in 2010. National surveys were fielded at the same time over the Internet (using an opt-in Internet panel), by telephone with live interviews (using a national Random Digit Dialing (RDD) sample of landlines and cell phones), and by mail (using a national sample of residential addresses). Each survey utilized a nearly identical questionnaire soliciting information across a range of political and social indicators, many of which can be validated with government data. Comparing the findings from the modes using a Total Survey Error approach, we demonstrate that a carefully executed opt-in Internet panel produces estimates that are as accurate as a telephone survey and that the two modes differ little in their estimates of other political indicators and their correlates.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“YouGov uses a model-based sampling approach to produce matched, weighted samples that generalize extremely well to the US adult population”
From Rape or Homicide: Which Is Worse? (2024) · cites this paper as Ansolabehere & Schaffner, 2014
Citations
Cites
What Americans Know about Politics and Why It Matters
Brooks, C., Carpini, M. X. D., & Keeter, S. (1997) · Contemporary Sociology · doi:10.2307/2655098 · Cited by 4,437
Nonresponse rates and nonresponse bias in household surveys
Groves, R. M. (2006) · Public Opinion Quarterly · doi:10.1093/poq/nfl033 · Cited by 2,420
What Americans know about politics and why it matters
(1996) · Choice Reviews Online · doi:10.5860/choice.34-0582 · Cited by 2,050
Krosnick, J. A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.537 · Cited by 1,852
National Surveys Via Rdd Telephone Interviewing Versus the Internet
Chang, L., & Krosnick, J. A. (2009) · Public Opinion Quarterly · doi:10.1093/poq/nfp075 · Cited by 955
Changes in Telephone Survey Nonresponse over the Past Quarter Century
Curtin, R., Presser, S., & Singer, E. (2005) · Public Opinion Quarterly · doi:10.1093/poq/nfi002 · Cited by 925
Yeager, D. S., Krosnick, J. A., Chang, L., Javitz, H. S., Levendusky, M. S., Simpser, A., & Wang, R. (2011) · Public Opinion Quarterly · doi:10.1093/poq/nfr020 · Cited by 758
Connections Between Internet Use and Political Efficacy, Knowledge, and Participation
Kenski, K., & Stroud, N. J. (2006) · Journal of Broadcasting & Electronic Media · doi:10.1207/s15506878jobem5002_1 · Cited by 751
Total Survey Error: Past, Present, and Future
Groves, R. M., & Lyberg, L. (2010) · Public Opinion Quarterly · doi:10.1093/poq/nfq065 · Cited by 594
Total Survey Error: Design, Implementation, and Evaluation
Biemer, P. P. (2010) · Public Opinion Quarterly · doi:10.1093/poq/nfq058 · Cited by 450
Research works printed but not linked: 3
- Rivers, 2009, Second thoughts about Internet surveys
- Chatt Cindy , Michael Dennis J. , Li Rick , and Pulliam Paul . 2003. Data collection mode effects controlling for sample origins in a panel survey: Telephone versus Internet. Paper presented at the Annual Meeting of the Midwest Chapter of the American Association for Public Opinion Research, Chicago.
- Groves, 2009, Survey Methodology
Cited by
Conspiracy Theories and the Paranoid Style(s) of Mass Opinion
Oliver, J. E., & Wood, T. J. (2014) · American Journal of Political Science · doi:10.1111/ajps.12084 · Cited by 877
Cornesse, C., Blom, A. G., Dutwin, D., Krosnick, J. A., De Leeuw, E. D., Legleye, S., Pasek, J., Pennay, D., Phillips, B., Sakshaug, J. W., et al. (2020) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smz041 · Cited by 418
Extreme weather events and climate change concern
Konisky, D. M., Hughes, L., & Kaylor, C. H. (2016) · Climatic Change · doi:10.1007/s10584-015-1555-3 · Cited by 367
Mass support for global climate agreements depends on institutional design
Bechtel, M. M., & Scheve, K. F. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1306374110 · Cited by 350
Ansolabehere, S., & Rivers, D. (2013) · Annual Review of Political Science · doi:10.1146/annurev-polisci-022811-160625 · Cited by 257
Self-citationMemmott, T., Carley, S., Graff, M., & Konisky, D. M. (2021) · Nature Energy · doi:10.1038/s41560-020-00763-9 · Cited by 238
Improving Climate Change Acceptance Among U.S. Conservatives Through Value-Based Message Targeting
Dixon, G., Hmielowski, J., & Ma, Y. (2017) · Science Communication · doi:10.1177/1075547017715473 · Cited by 222
Facebook news and (de)polarization: reinforcing spirals in the 2016 US election
Beam, M. A., Hutchens, M. J., & Hmielowski, J. D. (2018) · Information, Communication & Society · doi:10.1080/1369118x.2018.1444783 · Cited by 214
“YouGov’s sample-matching procedure has been empirically compared to random sampling procedures and found to exhibit little to no selection bias (see Ansolabehere & Rivers, 2013; Ansolabehere & Schaffner, 2014).”
MethodThe consequences of political innumeracy
Lawrence, E. D., & Sides, J. (2014) · Research & Politics · doi:10.1177/2053168014545414 · Cited by 130
“%) reported in Ansolabehere and Schaffner (2014).”
ResultGraham, A., Pickett, J. T., & Cullen, F. T. (2021) · Crime & Delinquency · doi:10.1177/0011128720977439 · Cited by 114
“…Current Population Survey (Simmons & Bobo, 2015).5 Other studies find that when compared against each other, there often are as many inconsistencies in results between different probability samples as between opt-in samples and probability samples (Ansolabehere & Schaffner, 2014; Bhutta, 2012).”
ResultReinforcing spirals of political discussion and affective polarization
Hutchens, M. J., Hmielowski, J. D., & Beam, M. A. (2019) · Communication Monographs · doi:10.1080/03637751.2019.1575255 · Cited by 113
“YouGov’s sample matching procedure has been empirically compared to random sampling procedures and found to exhibit little to no selection bias (Ansolabehere & Rivers, 2013; Ansolabehere & Schaffner, 2014; Rivers & Bailey, 2009).”
MethodPublic Support for Policies to Reduce School Shootings: A Moral-Altruistic Model
Burton, A. L., Pickett, J. T., Jonson, C. L., Cullen, F. T., & Burton, V. S. (2021) · Journal of Research in Crime and Delinquency · doi:10.1177/0022427820953202 · Cited by 86
When Common Identities Decrease Trust: An Experimental Study of Partisan Women
Klar, S. (2018) · American Journal of Political Science · doi:10.1111/ajps.12366 · Cited by 86
Political Chameleons: An Exploration of Conformity in Political Discussions
Carlson, T. N., & Settle, J. E. (2016) · Political Behavior · doi:10.1007/s11109-016-9335-y · Cited by 74
Silver, E., Goff, K., & Iceland, J. (2022) · Criminology · doi:10.1111/1745-9125.12303 · Cited by 59
Punitive Attitudes Toward Individuals Convicted of Sex Offenses: A Vignette Study
Socia, K. M., Rydberg, J., & Dum, C. P. (2021) · Justice Quarterly · doi:10.1080/07418825.2019.1683218 · Cited by 56
“See Ansolabehere and Schaffner (2014) for additional details on the sampling and survey methods.”
Drones and Support for the Use of Force
Walsh, J., & Schulzke, M. (2018) · doi:10.3998/mpub.9946611 · Cited by 49
Attributions as Anchors: How the Public Explains School Shootings and Why It Matters
Lee, H., Pickett, J. T., Burton, A. L., Cullen, F. T., Jonson, C. L., & Burton, Jr. (2022) · Justice Quarterly · doi:10.1080/07418825.2020.1769710 · Cited by 31
The consequences of personality biases in online panels for measuring public opinion
Valentino, N. A., Zhirkov, K., Hillygus, D. S., & Guay, B. (2021) · Public Opinion Quarterly · doi:10.1093/poq/nfaa026 · Cited by 29
What’s Not to Like? Facebook Page Likes Reveal Limited Polarization in Lifestyle Preferences
Praet, S., Guess, A. M., Tucker, J. A., Bonneau, R., & Nagler, J. (2022) · Political Communication · doi:10.1080/10584609.2021.1994066 · Cited by 22
“…the resulting inferences may be subject to qualification on the basis of known differences likely arising from the sampling frame of internet users (Ansolabehere & Schaffner, 2014) – namely, respondents in these samples tend to be more knowledgeable about, interested in and engaged with politics…”
BackgroundA measure of survey mode differences
Homola, J., Jackson, N., & Gill, J. (2016) · Electoral Studies · doi:10.1016/j.electstud.2016.06.010 · Cited by 21
Rape or Homicide: Which Is Worse?
Felson, R. B., & Silver, E. (2024) · Archives of Sexual Behavior · doi:10.1007/s10508-023-02799-w · Cited by 5
Moral Reactions to Sexual Relationships that Violate Age Norms
Felson, R. B., & Silver, E. (2024) · The Journal of Sex Research · doi:10.1080/00224499.2023.2192196 · Cited by 3
“YouGov uses a model-based sampling approach to produce matched, weighted samples that generalize extremely well to the U.S. adult population (Ansolabehere & Rivers, 2013; Ansolabehere & Schaffner, 2014; Vavreck & Rivers, 2008).”
Metadata
- Authors
- Ansolabehere, S., & Schaffner, B. F.
- Affiliations
- Harvard University (US)
- University of Massachusetts Amherst (US)
- Corresponding author
- Schaffner, Brian F.
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2014
- Volume
- 22
- Issue
- 3
- Pages
- 285-303
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W2092517937
- Semantic Scholar
- 87b83b77d780751adf895e5bb941f30107fc981b
- Cited by
- 573
- References
- 23
- Citations by year
- 2026: 25 · 2025: 34 · 2024: 40 · 2023: 40 · 2022: 84 · 2021: 52 · 2020: 48 · 2019: 59 · 2018: 54 · 2017: 39 · 2016: 38 · 2015: 25 · 2014: 23
- FWCI
- 135.17
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Political Analysis (publishedVersion)
- SSRN Electronic Journal (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Social Sciences › Political Science and International Relations › Electoral Systems and Political Participation
- Keywords
- survey mode, online surveys, telephone surveys, mail surveys, total survey error, measurement error, survey accuracy, political indicators, social indicators, random digit dialing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.