The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Survey Pretesting: Do Different Methods Produce Different Results?
Abstract
Pretesting is generally agreed to be an indispensable stage in survey questionnaire development, yet we know little about how well different pretesting methods identify particular types of problems. This study compared four pretesting methods using a single questionnaire in repeated trials of each. The four methods were conventional pretests, behavior coding, cognitive interviews, and expert panels. We developed a model-based coding scheme that classified problems as respondent-semantic, respondent-task, interviewer-task, or analysis. On average, expert panels were most productive in the number of problems identified. Conventional pretesting and behavior coding were the only methods to identify significant numbers of interviewer problems. By contrast, expert panels and cognitive interviews were the only methods to diagnose a nontrivial number of analysis problems. Expert panels and behavior coding were more consistent than the other methods in the numbers of problems identified across trials, as well as in their distribution of problem types. From the vantage point of the particular problems
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Survey Questions: Handcrafting the Standardized Questionnaire
Quarm, D., Converse, J. M., & Presser, S. (1988) · Teaching Sociology · doi:10.2307/1317544 · Cited by 766
Self-citationResearch Methods in Social Relations.
Davis, J. A., Selltiz, C., Jahoda, M., Deutsch, M., Cook, S. W., Chein, I., & Proshansky, H. M. (1960) · American Sociological Review · doi:10.2307/2092666 · Cited by 609
The Pretest in Survey Research: Issues and Preliminary Findings
Hunt, S. D., Sparkman, R. D., & Wilcox, J. B. (1982) · Journal of Marketing Research · doi:10.1177/002224378201900211 · Cited by 479
Research on Interviewing Techniques
Cannell, C. F., Miller, P. V., & Oksenberg, L. (1981) · Sociological Methodology · doi:10.2307/270748 · Cited by 436
The Sample Survey: Theory and Practice
Som, R. K., Warwick, D. P., & Lininger, C. A. (1976) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1402362 · Cited by 370
The Design and Understanding of Survey Questions.
Morton-Williams, J., & Belson, W. A. (1984) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981742 · Cited by 247
Campanelli, P., Martin, E. A., & Rothgeb, J. M. (1991) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348278 · Cited by 49
QUESTIONNAIRE DESIGN RESEARCH LABORATORY
Royston, P., Bercini, D., Sirken, M. G., & Mingay, D. J. (1985) · Cited by 19
Cited by
Krosnick, J. A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.537 · Cited by 1,852
Research synthesis: the practice of cognitive interviewing
Beatty, P. C., & Willis, G. B. (2007) · Public Opinion Quarterly · doi:10.1093/poq/nfm006 · Cited by 1,615
Pretesting survey instruments: An overview of cognitive methods
Collins, D. (2003) · Quality of Life Research · doi:10.1023/a:1023254226592 · Cited by 1,506
Methods for Testing and Evaluating Survey Questionnaires
(2004) · doi:10.1002/0471654728 · Cited by 855
The Science of Asking Questions
Schaeffer, N. C., & Presser, S. (2003) · Annual Review of Sociology · doi:10.1146/annurev.soc.29.110702.110112 · Cited by 621
Self-citationDesimone, L. M., & Le Floch, K. C. (2004) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737026001001 · Cited by 421
Designing and Conducting Health Surveys
Aday, L. A. (1998) · Journal For Healthcare Quality · doi:10.1097/01445442-199805000-00015 · Cited by 391
Holbrook, A. L., Krosnick, J. A., & Pfent, A. (2007) · Advances in Telephone Survey Methodology · doi:10.1002/9780470173404.ch23 · Cited by 380
Lindemann‐Matthies *, P. (2005) · International Journal of Science Education · doi:10.1080/09500690500038116 · Cited by 379
Curlin, F. A., Chin, M. H., Sellergren, S. A., Roach, C. J., & Lantos, J. D. (2006) · Medical Care · doi:10.1097/01.mlr.0000207434.12450.ef · Cited by 315
Research into Questionnaire Design: A Summary of the Literature
Lietz, P. (2010) · International Journal of Market Research · doi:10.2501/s147078530920120x · Cited by 313
Understanding and Improving the Validity of Self-Report of Parenting
Morsbach, S. K., & Prinz, R. J. (2006) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-006-0001-5 · Cited by 302
Revisiting Consumer Ethnocentrism: Review, Reconceptualization, and Empirical Testing
Siamagka, N.-T., & Balabanis, G. (2015) · Journal of International Marketing · doi:10.1509/jim.14.0085 · Cited by 289
An Examination of Questionnaire Evaluation by Expert Reviewers
Olson, K. (2010) · Field Methods · doi:10.1177/1525822x10379795 · Cited by 253
Holden, R. J., Scanlon, M. C., Patel, N. R., Kaushal, R., Escoto, K. H., Brown, R. L., Alper, S. J., Arnold, J. M., Shalaby, T. M., Murkowski, K., et al. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs.2008.028381 · Cited by 252
Curlin, F. A., Lawrence, R. E., Odell, S., Chin, M. H., Lantos, J. D., Koenig, H. G., & Meador, K. G. (2007) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2007.06122088 · Cited by 213
Ikart, E. M. (2019) · Asian Journal of Social Science Studies · doi:10.20849/ajsss.v4i2.565 · Cited by 140
A Summary of Survey Methodology Best Practices for Security and Privacy Researchers
Redmiles, E. M., Acar, Y., Fahl, S., & Mazurek, M. L. (2017) · Digital Repository at the University of Maryland · doi:10.13016/m22k2w · Cited by 50
Metadata
- Authors
- Presser, S., & Blair, J.
- Venue
- Sociological Methodology
- Publisher
- JSTOR
- Published
- 1994
- Volume
- 24
- Pages
- 73
- ISSN
- 0081-1750 (ISSN-L)
- OpenAlex
- W2316050936
- Cited by
- 274
- References
- 8
- FWCI
- 16.52
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.