The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Pretesting survey instruments: An overview of cognitive methods
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 2×
“Finally respondents may take their available or representative answer and adjust it in light of the survey context, the answer categories provided or the special circumstance of last week (see Sudman et al. [22] and Tourangeau et al. [6] for a review of Tversky and Kahneman’s research on judgment).”
MethodAnderson, J. R. (2013) · doi:10.4324/9781315799438 · Cited by 6,761
The Psychology of Survey Response
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000) · doi:10.1017/cbo9780511819322 · Cited by 4,734
Encoding specificity and retrieval processes in episodic memory.
Tulving, E., & Thomson, D. M. (1973) · Psychological Review · doi:10.1037/h0020071 · Cited by 4,540 · cited 1×
“Firstly, if the retrieval context is different to the original encoding context the respondent may not be able to recognise that the event took place or be able to recall the correct event [23].”
BackgroundJudgement under Uncertainty: Heuristics and Biases
Hirschman, E. C., Kahneman, D., Slovic, P., & Tversky, A. (1983) · Journal of Marketing Research · doi:10.2307/3151689 · Cited by 3,813
Response strategies for coping with the cognitive demands of attitude measures in surveys
Krosnick, J. A. (1991) · Applied Cognitive Psychology · doi:10.1002/acp.2350050305 · Cited by 2,460
Reconstruction of automobile destruction: An example of the interaction between language and memory
Loftus, E. F., & Palmer, J. C. (1974) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(74)80011-3 · Cited by 1,909
Thinking About Answers: The Application of Cognitive Processes to Survey Methodology
Sudman, S., Bradburn, N. M., & Schwarz, N. (1995) · Cited by 1,586
Research works printed but not linked: 11
- Fowler FJ. ImprovingSurvey Questions: Design and Evaluation. Applied Social Research Methods Series: 38. Thousand Oaks: Sage, 1995.
- Oksenberg L, Cannell C, Kalton G. New strategies for pretesting survey questions. J Off Stat 1991; 7(3): 349–365.
- Cannell CF, Oksenberg L, Converse JM (eds). Experiments in Interviewing Techniques: Field Experiments in Health Reporting, 1971-1977. Survey Research Centre, Institute of Social Research, University of Michigan. Ann Arbor, MI: University of Michigan, 1979.
- Aborn M. CASM revisited. In: Sirken MG, Herrmann DJ, Schechter S, Schwarz N, Tanur JM, Tourangeau R (eds), Cognition and Survey Research. New York: Wiley, 1999: 21–38.
- Tourangeau R. Cognitive sciences and survey methods. In: Jabine T, Straf M, Tanur J, Tourangeau R (eds). Cognitive Aspects of Survey Methodology: Building a Bridge Between the Disciplines. Washington, DC: National Academy Press, 1984: 73–100.
- Esposito J, Jobe J. A General Model of the Survey Interaction Process. Bureau of the Census Seventh Annual Research Conference Proceedings. Washington DC: US Bureau of the Census, 1991: 537–560.
- Schober MF. Making sense of questions: an interactional approach. In: Sirken, MG, Herrmann DJ, Schechter S, Schwarz N, Tanur JM, Tourangeau R (eds), Cognition and Survey Research. New York: Wiley, 1999: 77–93.
- Czaja, R. Questionnaire pretesting comes of age. Market Bull 1998; 9: 52–66.
- Schwarz N. Questionnaire design: The rocky road from concepts to answers. In: LybergL, Biemer P, Collins M, de Leeuw E, Dippo C, Schwarz N, Trewin D (eds), Survey Measurement and Process Quality. New York: Wiley, 1997: 29–45.
- Willis GB. Cognitive Interviewing and Questionnaire Design: A training manual. Hyattsville, MD: National Centre for Health Statistics, 1994.
- Willis GB, DeMaio T, Harris-Kojetin B. Is the bandwagon headed to the methodological promised land? Evaluating the validity of cognitive interviewing techniques. In: Sirken MG, Herrmann DJ, Schechter S, Schwarz N, Tanur JM, Tourangeau R (eds), Cognition and Survey Research. New York: Wiley, 1999: 133–153.
Cited by
The COMET Handbook: version 1.0
Williamson, P. R., Altman, D. G., Bagley, H., Barnes, K. L., Blazeby, J. M., Brookes, S. T., Clarke, M., Gargon, E., Gorst, S., Harman, N., et al. (2017) · Trials · doi:10.1186/s13063-017-1978-4 · Cited by 2,011
“Where the qualitative work has a very specific focus; for example, informing an online Delphi survey for a particular stakeholder group, interviewing might focus on a prototype of the survey and use think-aloud techniques [103, 104] to explore how stakeholders interpret the outcomes and how the phrasing might be refin…”
BackgroundA guide for the design and conduct of self-administered surveys of clinicians
Burns, K. E. A., Duffett, M., Kho, M. E., Meade, M. O., Adhikari, N. K. J., Sinuff, T., Cook, D. J., & for the ACCADEMY Group (2008) · Canadian Medical Association Journal · doi:10.1503/cmaj.080372 · Cited by 1,354
(2011) · Amsterdam UMC (VU Amsterdam) - Institutional Repository · Cited by 1,209
Epstein, J., Santo, R. M., & Guillemin, F. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.021 · Cited by 990
RETROSPECTIVE REPORTS IN ORGANIZATIONAL RESEARCH: A REEXAMINATION OF RECENT EVIDENCE.
MILLER, C. C., CARDINAL, L. B., & GLICK, W. H. (1997) · Academy of Management Journal · doi:10.2307/257026 · Cited by 898
"Any other comments?" Open questions on questionnaires – a bane or a bonus to research?
O'Cathain, A., & Thomas, K. J. (2004) · BMC Medical Research Methodology · doi:10.1186/1471-2288-4-25 · Cited by 646
Ardito, L., Raby, S., Albino, V., & Bertoldi, B. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2020.09.022 · Cited by 485
Matza, L. S., Patrick, D. L., Riley, A. W., Alexander, J. J., Rajmil, L., Pleil, A. M., & Bullinger, M. (2013) · Value in Health · doi:10.1016/j.jval.2013.04.004 · Cited by 477
PRO development: rigorous qualitative research as the crucial foundation
Lasch, K. E., Marquis, P., Vigneux, M., Abetz, L., Arnould, B., Bayliss, M., Crawford, B., & Rosa, K. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9677-6 · Cited by 461
“There is also extensive literature on the psychology of survey research that addresses how respondents answer items on a questionnaire [9–11, 24–28].”
Backgroundvan der Vaart, R., van Deursen, A. J., Drossaert, C. H., Taal, E., van Dijk, J. A., & van de Laar, M. A. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1840 · Cited by 404
“To this aim, qualitative research might provide more insight into the basis for participants’ answers—for example, having people fill out the eHEALS with techniques such as cognitive interviewing or thinking-aloud methods [43,44].”
BackgroundSwami, V., & Barron, D. (2019) · Body Image · doi:10.1016/j.bodyim.2018.08.014 · Cited by 454
Rodrigues, I. B., Adachi, J. D., Beattie, K. A., & MacDermid, J. C. (2017) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-017-1914-5 · Cited by 440
Parsons, K., McCormac, A., Butavicius, M., Pattinson, M., & Jerram, C. (2014) · Computers & Security · doi:10.1016/j.cose.2013.12.003 · Cited by 437
Best Practices for Survey Research Reports: A Synopsis for Authors and Reviewers
Draugalis, J. R., Coons, S. J., & Plaza, C. M. (2008) · American Journal of Pharmaceutical Education · doi:10.5688/aj720111 · Cited by 434
Opel, D. J., Mangione-Smith, R., Taylor, J. A., Korfiatis, C., Wiese, C., Catz, S., & Martin, D. P. (2011) · Human Vaccines · doi:10.4161/hv.7.4.14120 · Cited by 432
Digital Leadership Skills and Associations with Psychological Well-Being
Zeike, S., Bradbury, K., Lindert, L., & Pfaff, H. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16142628 · Cited by 392
Bird, D. K. (2009) · Natural Hazards and Earth System Sciences · doi:10.5194/nhess-9-1307-2009 · Cited by 364
National Comorbidity Survey Replication Adolescent Supplement (NCS-A): I. Background and Measures
Merikangas, K. R., Avenevoli, S., Costello, E. J., Koretz, D., & Kessler, R. C. (2009) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/chi.0b013e31819996f1 · Cited by 351
The analysis and interpretation of cognitive interviews for instrument development
Knafl, K., Deatrick, J., Gallo, A., Holcombe, G., Bakitas, M., Dixon, J., & Grey, M. (2007) · Research in Nursing & Health · doi:10.1002/nur.20195 · Cited by 341
The Pitfalls and Promise of Focus Groups as a Data Collection Method
Cyr, J. (2016) · Sociological Methods & Research · doi:10.1177/0049124115570065 · Cited by 317
Family Influence and Digital Business Model Innovation: The Enabling Role of Dynamic Capabilities
Soluk, J., Miroshnychenko, I., Kammerlander, N., & De Massis, A. (2021) · Entrepreneurship Theory and Practice · doi:10.1177/1042258721998946 · Cited by 287
Jones, R. K., & Kost, K. (2007) · Studies in Family Planning · doi:10.1111/j.1728-4465.2007.00130.x · Cited by 271
Abou-Foul, M., Ruiz-Alba, J. L., & López-Tenorio, P. J. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2022.113609 · Cited by 253
Instrumentation issues in implementation science
Martinez, R. G., Lewis, C. C., & Weiner, B. J. (2014) · Implementation Science · doi:10.1186/s13012-014-0118-8 · Cited by 243
“Items from the initial pool should be administered to a small, representative sample of respondents to assess face validity, identify missing items, and assess whether the language is appropriate, potentially through a thinkaloud technique [35].”
BackgroundUnited States national pain strategy for population research: concepts, definitions, and pilot data.
Von Korff, M., Scher, A. I., Helmick, C., Carter-Pokras, O., Dodick, D. W., Goulet, J., Hamill-Ruth, R., LeResche, L., Porter, L., Tait, R., et al. (2016) · The Journal of Pain · doi:10.1016/j.jpain.2016.06.009 · Cited by 229
Brédart, A., Marrel, A., Abetz-Webb, L., Lasch, K., & Acquadro, C. (2014) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-12-15 · Cited by 195
“Cognitive interviews are carried out using two specific techniques: the think aloud technique, whereby patients are invited to express out loud their thought process while reading and answering the item; or the probe technique which essentially asks the patient to paraphrase items [34-36].”
MethodPatient and Public Involvement in Patient-Reported Outcome Measures: Evolution Not Revolution
Staniszewska, S., Haywood, K. L., Brett, J., & Tutton, L. (2012) · The Patient: Patient-Centered Outcomes Research · doi:10.2165/11597150-000000000-00000 · Cited by 196
McGrath, C., Cerratto Pargman, T., Juth, N., & Palmgren, P. J. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100139 · Cited by 195
Wolf, S., Aber, J. L., Behrman, J. R., & Tsinigo, E. (2019) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2018.1517199 · Cited by 154
“First, we conducted five cognitive interviews with teachers to assess whether they understood each question, both consistently across constructs and in the way the item was intended (Collins, 2003).”
MethodBrownell, S. E., Price, J. V., & Steinman, L. (2013) · Advances in Physiology Education · doi:10.1152/advan.00138.2012 · Cited by 135
Instrument Selection Using the OMERACT Filter 2.1: The OMERACT Methodology
Beaton, D. E., Maxwell, L. J., Shea, B. J., Wells, G. A., Boers, M., Grosskleg, S., Bingham, C. O., Conaghan, P. G., D’Agostino, M. A., de Wit, M. P., et al. (2019) · The Journal of Rheumatology · doi:10.3899/jrheum.181218 · Cited by 96
Malm, A., Tinghög, P., Narusyte, J., & Saboonchi, F. (2020) · Conflict and Health · doi:10.1186/s13031-019-0246-5 · Cited by 89
The PROactive innovative conceptual framework on physical activity
Dobbels, F., de Jong, C., Drost, E., Elberse, J., Feridou, C., Jacobs, L., Rabinovich, R., Frei, A., Puhan, M. A., de Boer, W. I., et al. (2014) · European Respiratory Journal · doi:10.1183/09031936.00004814 · Cited by 67
Fischer, S. L., Sinden, K. E., & MacPhee, R. S. (2017) · Applied Ergonomics · doi:10.1016/j.apergo.2017.06.021 · Cited by 60
Examining how rural ecological contexts influence children’s early learning opportunities
Iruka, I. U., DeKraai, M., Walther, J., Sheridan, S. M., & Abdel-Monem, T. (2020) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2019.09.005 · Cited by 51
A Summary of Survey Methodology Best Practices for Security and Privacy Researchers
Redmiles, E. M., Acar, Y., Fahl, S., & Mazurek, M. L. (2017) · Digital Repository at the University of Maryland · doi:10.13016/m22k2w · Cited by 50
“[a particular term] mean to you?” Cognitive interviewing is broadly recommended as a necessary and highly-important pre-testing measure in the survey literature [11], [17], [40], [103], [122], [160].”
BackgroundExamining the educational impact of the mini-CEX: a randomised controlled study
Martinsen, S. S. S., Espeland, T., Berg, E. A. R., Samstad, E., Lillebo, B., & Slørdahl, T. S. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02670-3 · Cited by 45
Watson, E. K., Brett, J., Hay, H., Witwicki, C., Perris, A., Poots, A. J., Sizmur, S., Soonawalla, Z., & Tallett, A. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-032681 · Cited by 42
Content Validity of the Credibility and Expectancy Questionnaire in a Pain Rehabilitation Setting
Mertens, V., Moser, A., Verbunt, J., Smeets, R., & Goossens, M. (2017) · Pain Practice · doi:10.1111/papr.12543 · Cited by 37
Optimizing the Communication with Cancer Patients During the COVID-19 Pandemic: Patient Perspectives
Tashkandi, E., BaAbdullah, M., Zeeneldin, A., AlAbdulwahab, A., Elemam, O., Elsamany, S., Alfayez, M., Dabash, Y., Khayat, E., Hassanin, F., et al. (2020) · Patient Preference and Adherence · doi:10.2147/ppa.s263022 · Cited by 30
Lövestam, E., Vivanti, A., Steiber, A., Boström, A.-M., Devine, A., Haughey, O., Kiss, C. M., Lang, N. R., Lieffers, J., Lloyd, L., et al. (2019) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2018.09.004 · Cited by 30
Bilardi, J. E., Sharp, G., Payne, S., & Temple-Smith, M. J. (2021) · Women and Birth · doi:10.1016/j.wombi.2020.06.011 · Cited by 29
Ingeman, K., Frostholm, L., Frydendal, D. H., Wright, K. D., Lockhart, E., Garralda, M. E., Kangas, M., & Rask, C. U. (2021) · Nordic Journal of Psychiatry · doi:10.1080/08039488.2021.1900389 · Cited by 20
Taylor, J. C., Heuer, H. W., Clark, A. L., Wise, A. B., Manoochehri, M., Forsberg, L., Mester, C., Rao, M., Brushaber, D., Kramer, J., et al. (2023) · Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring · doi:10.1002/dad2.12423 · Cited by 20
Development of a Caregiver-Reported experience measure for pediatric Hospital-to-Home transitions
Desai, A. D., Jacob‐Files, E. A., Lowry, S. J., Opel, D. J., Mangione‐Smith, R., Britto, M. T., & Howard, W. J. (2018) · Health Services Research · doi:10.1111/1475-6773.12864 · Cited by 19
The development, validation, and feasibility of the Experienced Coercion Scale
Nyttingnes, O., Rugkåsa, J., Holmén, A., & Ruud, T. (2017) · Psychological Assessment · doi:10.1037/pas0000404 · Cited by 15
Carlson, G. C., Kelly, M. R., Grinberg, A. M., Mitchell, M., McGowan, S. K., Culver, N. C., Kay, M., Alessi, C. A., Washington, D. L., Yano, E. M., et al. (2021) · Behavioral Sleep Medicine · doi:10.1080/15402002.2020.1846537 · Cited by 11
Petrič, G. (2024) · International Journal of Technology and Human Interaction · doi:10.4018/ijthi.349225 · Cited by 10
Metadata
- Authors
- Collins, D.
- Venue
- Quality of Life Research
- Published
- 2003-05
- Volume
- 12
- Issue
- 3
- Pages
- 229-238
- ISSN
- 0962-9343 (ISSN-L)
- 1573-2649
- OpenAlex
- W2117891525
- Cited by
- 1,506
- References
- 36
- FWCI
- 28.27
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.