Comparing the use of open and closed questions for Web-based measures of the continued-influence effect
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Open-ended questions, in which participants write or type their responses, are used in many areas of the behavioral sciences. Although effective in the lab, they are relatively untested in online experiments, and the quality of responses is largely unexplored. Closed-ended questions are easier to use online because they generally require only single key- or mouse-press responses and are less cognitively demanding, but they can bias the responses. We compared the data quality obtained using open and closed response formats using the continued-influence effect (CIE), in which participants read a series of statements about an unfolding event, one of which is unambiguously corrected later. Participants typically continue to refer to the corrected misinformation when making inferential statements about the event. We implemented this basic procedure online (Exp. 1A, n = 78), comparing standard open-ended responses to an alternative procedure using closed-ended responses (Exp. 1B, n = 75). Finally, we replicated these findings in a larger preregistered study (Exps. 2A and 2B, n = 323). We observed the CIE in all conditions: Participants continued to refer to the misinformation following a correction, and their references to the target misinformation were broadly similar in number across open- and closed-ended questions. We found that participants' open-ended responses were relatively detailed (including an average of 75 characters for inference questions), and almost all responses attempted to address the question. The responses were faster, however, for closed-ended questions. Overall, we suggest that with caution it may be possible to use either method for gathering CIE data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448
The Nature of Recollection and Familiarity: A Review of 30 Years of Research
Yonelinas, A. P. (2002) · Journal of Memory and Language · doi:10.1006/jmla.2002.2864 · Cited by 3,840 · cited 1×
“There is also empirical evidence to suggest that open- and closed-end responses are supported by different cognitive (Frew, Whynes, & Wolstenholme, 2003; Frew, Wolstenholme, & Whynes, 2004) or memory (Khoe, Kroll, Yonelinas, Dobbins, & Knight, 2000; see Yonelinas, 2002, for a review) processes.”
BackgroundBeyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957
Misinformation and Its Correction: Continued Influence and Successful Debiasing
Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N., & Cook, J. (2012) · Psychological Science in the Public Interest · doi:10.1177/1529100612451018 · Cited by 2,920
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617
Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014) · American Journal of Political Science · doi:10.1111/ajps.12081 · Cited by 951
Debunking: A Meta-Analysis of the Psychological Efficacy of Messages Countering Misinformation
Chan, M.-P. S., Jones, C. R., Hall Jamieson, K., & Albarracín, D. (2017) · Psychological Science · doi:10.1177/0956797617714579 · Cited by 898
Germine, L., Nakayama, K., Duchaine, B. C., Chabris, C. F., Chatterjee, G., & Wilmer, J. B. (2012) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0296-9 · Cited by 669 · cited 2×
“…the justification for this shift has been the finding that the data quality fromWeb-based studies is comparable to that obtained in the lab: The vast majority of Web-based studies have replicated existing findings (e.g., Crump, McDonnell, & Gureckis, 2013; Germine et al., 2012; Zwaan et al., 2017).”
ResultKnowledge does not protect against illusory truth.
Fazio, L. K., Brashier, N. M., Payne, B. K., & Marsh, E. J. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000098 · Cited by 598
Cited by
Nagel, M.-T., Zlatkin-Troitschanskaia, O., & Fischer, J. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.914857 · Cited by 27
Continued Influence of Misinformation and the Information Disorder
Tay, L. Q., & Ecker, U. K. H. (2025) · The Remaking of Memory in the Age of the Internet and Social Media · doi:10.1093/oso/9780197661260.003.0009 · Cited by 12
Fake News on Social Media: Predicting Which Media Format Influences Fake News Most on Facebook
Demuyakor, J., & Opata, E. M. (2022) · Journal of Intelligent Communication · doi:10.54963/jic.v2i1.56 · Cited by 9
“According to Connor Desai, S., & Reimers [46], the unrestricted responses also often facilitate confirmation bias by establishing that the portrayals of the news story are often aligned with an individual’s prior associations, for instance, one is more likely to believe how a terrorist is perceived or how a political…”
Background
Metadata
- Authors
- Connor Desai, S., & Reimers, S.
- Venue
- Behavior Research Methods
- Published
- 2018-06-25
- Volume
- 51
- Issue
- 3
- Pages
- 1426-1440
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2806851250
- Cited by
- 127
- References
- 13
- FWCI
- 18.23
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Comparing the use of open and closed questions for Web-based measures of the continued-influence effect. https://app.cochat.ai/papers/10.3758/s13428-018-1066-z
- The paper
- Connor Desai, S., & Reimers, S. (2019). Comparing the use of open and closed questions for Web-based measures of the continued-influence effect. Behavior Research Methods. https://doi.org/10.3758/s13428-018-1066-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.