Submitting surveys via a conversational interface: An evaluation of user acceptance and approach effectiveness
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Conversational interfaces are currently on the rise: more and more applications rely on a chat-like interaction pattern to increase their acceptability and to improve user experience. Also in the area of questionnaire design and administration, interaction design is increasingly looked at as an important ingredient of a digital solution. For those reasons, we designed and developed a conversational survey tool to administer questionnaires with a colloquial form through a chat-like Web interface. In this paper, we present the evaluation results of our approach, taking into account both the user point of view - by assessing user acceptance and preferences in terms of survey compilation experience - and the survey design perspective - by investigating the effectiveness of a conversational survey in comparison to a traditional questionnaire. We show that users clearly appreciate the conversational form and prefer it over a traditional approach and that, from a data collection point of view, the conversational method shows the same reliability and a higher response quality with respect to a traditional questionnaire.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
Prolific.ac—A subject pool for online experiments
Palan, S., & Schitter, C. (2018) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2017.12.004 · Cited by 3,450
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957 · cited 1×
“…oriented to atomic task execution, Prolific is designed for researchers and their needs and specifically suitable for social and economic science experiments (Peer et al. (2017)); moreover, the “workers” panel is quite varied and more than half of the participants have a bachelor degree or higher 9 .”
BackgroundSelf-disclosure in computer-mediated communication: The role of self-awareness and visual anonymity
Joinson, A. N. (2001) · European Journal of Social Psychology · doi:10.1002/ejsp.36 · Cited by 1,458
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176 · cited 2×
“Avoiding the use of NLU, our goal is to overcome the limits in user experience when the system does not fully “understand” user’s utterances, thus failing to meet user expectations, as illustrated by Luger and Sellen (2016).”
BackgroundMethodUnderstanding dynamics between initial trust and usage intentions of mobile banking
Kim, G., Shin, B., & Lee, H. G. (2009) · Information Systems Journal · doi:10.1111/j.1365-2575.2007.00269.x · Cited by 1,137
Evaluating and Informing the Design of Chatbots
Jain, M., Kumar, P., Kota, R., & Patel, S. N. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196735 · Cited by 376
Reporting Mental Health Symptoms: Breaking Down Barriers to Care with Virtual Human Interviewers
Lucas, G. M., Rizzo, A., Gratch, J., Scherer, S., Stratou, G., Boberg, J., & Morency, L.-P. (2017) · Frontiers in Robotics and AI · doi:10.3389/frobt.2017.00051 · Cited by 308 · cited 1×
“Even more advanced interaction modes are adopted to conduct interviews: virtual agents in the form of avatars are employed to increase self-disclosure and to create rapport by Mell et al. (2017), and show also their benefits in the case of mental health reporting, as in Lucas et al. (2017).”
BackgroundTrusting Virtual Agents: The Effect of Personality
Zhou, M. X., Mark, G., Li, J., & Yang, H. (2019) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3232077 · Cited by 176
Cited by
Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
Wei, J., Kim, S., Jung, H., & Kim, Y.-H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637364 · Cited by 84
“Lastly, other commercial chatbot frameworks, such as Dialogflow [33] and IBM Watson [36], also allow developers to build rule-based chatbots with GUIs [98].”
BackgroundMethodResultInfluentialJabir, A. I., Martinengo, L., Lin, X., Torous, J., Subramaniam, M., & Tudor Car, L. (2023) · Journal of Medical Internet Research · doi:10.2196/44548 · Cited by 63
“A recent study suggested that a conversational survey collected directly by a CA is a reliable alternative to traditional surveys and may lead to improved response quality [64].”
Wei, J., Tag, B., Trippas, J. R., Dingler, T., & Kostakos, V. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517432 · Cited by 46
Understanding How to Administer Voice Surveys through Smart Speakers
Wei, J., Jiang, W., Wang, C., Yu, D., Goncalves, J., Dingler, T., & Kostakos, V. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555606 · Cited by 9
“Similarly, Celino and Calegari [8] showed that submitting responses to conversational surveys can achieve the same reliability and a higher response quality with respect to traditional web surveys.”
Background
Metadata
- Authors
- Celino, I., & Re Calegari, G.
- Venue
- International Journal of Human-Computer Studies
- Publisher
- Elsevier BV
- Published
- 2020-07
- Volume
- 139
- Pages
- 102410
- ISSN
- 1071-5819 (ISSN-L)
- OpenAlex
- W3005176269
- Cited by
- 63
- References
- 11
- FWCI
- 13.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Submitting surveys via a conversational interface: An evaluation of user acceptance and approach effectiveness. https://app.cochat.ai/papers/10.1016/j.ijhcs.2020.102410
- The paper
- Celino, I., & Re Calegari, G. (2020). Submitting surveys via a conversational interface: An evaluation of user acceptance and approach effectiveness. International Journal of Human-Computer Studies. https://doi.org/10.1016/j.ijhcs.2020.102410
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.