User Experience of a Chatbot Questionnaire Versus a Regular Computer Questionnaire: Prospective Comparative Study
Open accessPublished version of a preprint
Abstract
Background: Respondent engagement of questionnaires in health care is fundamental to ensure adequate response rates for the evaluation of services and quality of care. Conventional survey designs are often perceived as dull and unengaging, resulting in negative respondent behavior. It is necessary to make completing a questionnaire attractive and motivating. Objective: The aim of this study is to compare the user experience of a chatbot questionnaire, which mimics intelligent conversation, with a regular computer questionnaire. Methods: The research took place at the preoperative outpatient clinic. Patients completed both the standard computer questionnaire and the new chatbot questionnaire. Afterward, patients gave their feedback on both questionnaires by the User Experience Questionnaire, which consists of 26 terms to score. Results: The mean age of the 40 included patients (25 [63%] women) was 49 (SD 18-79) years; 46.73% (486/1040) of all terms were scored positive for the chatbot. Patients preferred the computer for 7.98% (83/1040) of the terms and for 47.88% (498/1040) of the terms there were no differences. Completion (mean time) of the computer questionnaire took 9.00 minutes by men (SD 2.72) and 7.72 minutes by women (SD 2.60; P=.148). For the chatbot, completion by men took 8.33 minutes (SD 2.99) and by women 7.36 minutes (SD 2.61; P=.287). Conclusions: Patients preferred the chatbot questionnaire over the computer questionnaire. Time to completion of both questionnaires did not differ, though the chatbot questionnaire on a tablet felt more rapid compared to the computer questionnaire. This is an important finding because it could lead to higher response rates and to qualitatively better responses in future questionnaires.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Patients prefer interacting with chatbots for self-reported measures.
“There is growing evidence that technology can improve help-seeking behaviour [Johnson et al., 2022] and that patients prefer interacting with chatbots for self-reported measures [Te Pas et al., 2020] and discussing sensitive topics [Bhakta et al., 2014].”
From Closing the accessibility gap to mental health treatment with a personalized self-referral chatbot (2024) · cites this paper as Te Pas et al., 2020
Findings from previous research suggest that digital technologies may facilitate help-seeking for mental health and make it easier for people to disclose personal information.
“Findings from previous research suggest that digital technologies may facilitate help-seeking for mental health [Johnson et al., 2022] and make it easier for people to disclose personal information [Te Pas et al., 2020, Bhakta et al., 2014].”
From Closing the accessibility gap to mental health treatment with a personalized self-referral chatbot (2024) · cites this paper as Te Pas et al., 2020, Bhakta et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.361 · Cited by 2,817 · see citing works
Construction and Evaluation of a User Experience Questionnaire
Laugwitz, B., Held, T., & Schrepp, M. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89350-9_6 · Cited by 2,114 · see citing works
Is it all a game? Understanding the principles of gamification
Robson, K., Plangger, K. A., Kietzmann, J., McCarthy, I. P., & PITT, L. F. (2015) · Business Horizons · doi:10.1016/j.bushor.2015.03.006 · Cited by 837 · see citing works
Nadarzynski, T., Miles, O., Cowie, A., & Ridge, D. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619871808 · Cited by 810 · see citing works
Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios
Schrepp, M., Hinderks, A., & Thomaschewski, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-07668-3_37 · Cited by 560 · see citing works
Conversational Agents in Health Care: Scoping Review and Conceptual Analysis
Tudor Car, L., Dhinagaran, D. A., Kyaw, B. M., Kowatsch, T., Joty, S., Theng, Y.-L., & Atun, R. (2020) · Journal of Medical Internet Research · doi:10.2196/17158 · Cited by 553 · cited 1× · see citing works
“Because a chatbot is suitable to meet the aforementioned criteria to improve response rates of questionnaires, this prospective preliminary study will focus on the usage of a chatbot [13,16].”
BackgroundSurvey on Chatbot Design Techniques in Speech Conversation Systems
A., S., & John, D. (2015) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2015.060712 · Cited by 479 · see citing works
Methods to increase response rates to postal questionnaires
Edwards, P. J., Roberts, I., Clarke, M. J., DiGuiseppi, C. G., Wentz, R., Kwan, I., Cooper, R., Felix, L. M., & Pratap, S. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000008.pub3 · Cited by 390 · see citing works
Physicians’ perceptions of chatbots in health Care: cross-sectional web-based survey
Palanica, A., Flaschner, P., Thommandram, A., Li, M., & Fossat, Y. (2019) · Journal of Medical Internet Research · doi:10.2196/12887 · Cited by 384 · see citing works
Evaluating and Informing the Design of Chatbots
Jain, M., Kumar, P., Kota, R., & Patel, S. N. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196735 · Cited by 376 · see citing works
Sahlqvist, S., Song, Y., Bull, F., Adams, E. J., Preston, J. M., & Ogilvie, D. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-62 · Cited by 362 · see citing works
Nakash, R. A., Hutton, J. L., Jørstad‐Stein, E. C., Gates, S., & Lamb, S. E. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-5 · Cited by 360 · see citing works
Greer, S., Ramo, D., Chang, Y.-J., Fu, M., Moskowitz, J., & Haritatos, J. (2019) · JMIR mHealth and uHealth · doi:10.2196/15018 · Cited by 265 · see citing works
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
Brewster, S. A., Fitzpatrick, G., Cox, A. J., & Kostakos, V. (2019) · Cited by 238 · see citing works
Myths and Realities of Respondent Engagement in Online Surveys
Guin, T. D.-L., Baker, R., Mechling, J., & Ruyle, E. (2012) · International Journal of Market Research · doi:10.2501/ijmr-54-5-613-633 · Cited by 237 · see citing works
Raine, R., Fitzpatrick, R. M., Barratt, H., Bevan, G., Black, N. A., Boaden, R., Bower, P., Campbell, M., Denis, J., Devers, K. J., et al. (2016) · Health Services and Delivery Research · doi:10.3310/hsdr04160 · Cited by 187 · see citing works
Barriers to Completion of Patient Reported Outcome Measures
Schamber, E. M., Takemoto, S. K., Chenok, K. E., & Bozic, K. J. (2013) · The Journal of Arthroplasty · doi:10.1016/j.arth.2013.06.025 · Cited by 157 · see citing works
Value-based Healthcare: Patient-reported Outcomes in Clinical Decision Making
Baumhauer, J. F., & Bozic, K. J. (2016) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-016-4813-4 · Cited by 151 · see citing works
Effects of gamification on participation and data quality in a real-world market research domain
Cechanowicz, J., Gutwin, C., Brownell, B., & Goodfellow, L. (2013) · Proceedings of the First International Conference on Gameful Design, Research, and Applications · doi:10.1145/2583008.2583016 · Cited by 103 · see citing works
Peters, M., Crocker, H., Jenkinson, C., Doll, H., & Fitzpatrick, R. M. (2014) · BMJ Open · doi:10.1136/bmjopen-2013-003968 · Cited by 91 · see citing works
Gamification of Online Surveys: Design Process, Case Study, and Evaluation
Harms, J., Biegler, S., Wimmer, C., Kappel, K., & Grechenig, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22701-6_16 · Cited by 53 · see citing works
From the Australian Commission on Safety and Quality in Health Care
Bedford, G., Allen, S., & Slawomirski, L. (2012) · The Medical Journal of Australia · doi:10.5694/mja12.10229 · Cited by 53 · see citing works
Ho, A., Purdie, C., Tirosh, O., & Tran, P. (2019) · Patient Related Outcome Measures · doi:10.2147/prom.s162476 · Cited by 44 · see citing works
Factors influencing the response to postal questionnaire surveys about respiratory symptoms
Hazell, M. L., Morris, J., Linehan, M. F., FRANK, P. I., & Frank, T. L. (2008) · Primary Care Respiratory Journal · doi:10.3132/pcrj.2009.00001 · Cited by 38 · see citing works
Toepoel, V., & Lugtig, P. (2022) · Social Science Computer Review · doi:10.1177/0894439318784882 · Cited by 30 · see citing works
Patient-reported outcome measures and the evaluation of services
Gibbons, E. J., Black, N. A., Fallowfield, L. J., Newhouse, R. P., & Fitzpatrick, R. M. (2016) · Sussex Research Online (University of Sussex) · Cited by 25 · see citing works
Gaming for Respondents: A Test of the Impact of Gamification on Completion Rates
Warnock, S., & Gantz, J. S. (2017) · International Journal of Market Research · doi:10.2501/ijmr-2017-005 · Cited by 9 · see citing works
[A nudge in the right direction? Recognition and use of nudging in the medical profession].
Nagtegaal, R. (2020) · PubMed · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [19]Baumhauer JF Bozic KJ Value-based Healthcare: Patient-reported Outcomes in Clinical Decision Making Clin Orthop Relat Res 2016 6 474 6 1375 8 10.1007/s11999-016-4813-4 27052020 27052020 PMC4868147 · DOI
- [25]Cambridge Dictionary 2020-06-30 https://dictionary.cambridge.org/dictionary/english/nudging
Cited by
Sorted by citations
Chen, Y., Jensen, S., Albert, L. J., Gupta, S., & Lee, T. (2023) · Information Systems Frontiers · doi:10.1007/s10796-022-10291-4 · Cited by 685 · see citing works
Closing the accessibility gap to mental health treatment with a personalized self-referral chatbot
Habicht, J., Viswanathan, S., Carrington, B., Hauser, T. U., Harper, R., & Rollwage, M. (2024) · Nature Medicine · doi:10.1038/s41591-023-02766-x · Cited by 132 · see citing works
Li, B., Gross, O., Crampton, N., Kapoor, M., Tauseef, S., Jain, M., Truong, K. N., & Mariakakis, A. T. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641913 · Cited by 40 · see citing works
AI-experiments in education: An AI-driven randomized controlled trial for higher education research
Cingillioglu, I., Gal, U., & Prokhorov, A. B. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12633-y · Cited by 34 · see citing works
Liu, D., Zhang, Y., Zhao, B., Ma, S., Shi, C., & Ma, X. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714196 · Cited by 16 · see citing works
The way you assess matters: User interaction design of survey chatbots for mental health
Jin, Y., Chen, L., Zhao, X., & Cai, W. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103290 · Cited by 20 · see citing works
de Queiroz, D. A., Passarello, R. S., Fé, V. V. D. M., Rossini, A., da Silveira, É. F., de Queiroz, E. A. I. F., & da Costa, C. A. (2023) · Healthcare Analytics · doi:10.1016/j.health.2023.100257 · Cited by 14 · see citing works
Nguyen, M. H., Sedoc, J., & Taylor, C. O. (2024) · Journal of Medical Internet Research · doi:10.2196/55164 · Cited by 14 · see citing works
Multilingual Framework for Risk Assessment and Symptom Tracking (MRAST)
Šafran, V., Lin, S. M., Nateqi, J., Martin, A. G., Smrke, U., Ariöz, U., Plohl, N., Rojc, M., Bēma, D., Chávez, M., et al. (2024) · Sensors · doi:10.3390/s24041101 · Cited by 13 · see citing works
Li, B., Tauseef, S., Truong, K. N., & Mariakakis, A. T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713613 · Cited by 13 · see citing works
Vukovac, D. P., Horvat, A., & Čižmešija, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77943-6_14 · Cited by 11 · see citing works
Sorici, A., Băjenaru, L., Mocanu, I. G., Florea, A. M., Tsakanikas, P., Ribigan, A. C., Pedullà, L., & Bougea, A. M. (2023) · Healthcare · doi:10.3390/healthcare11192656 · Cited by 10 · see citing works
ChatGPT Role in a Medical Survey
AlSamhori, A. R. F., AlSamhori, J. F., & AlSamhori, A. F. (2023) · High Yield Medical Reviews · doi:10.59707/hymrtffp5435 · Cited by 12 · see citing works
Cuesta, S. A., Blanes-Selva, V., Conejero, J. A., Portolés, M. G., & García-Gómez, M. (2021) · Informatics for Health and Social Care · doi:10.1080/17538157.2021.1923501 · Cited by 11 · see citing works
How the Wakamola chatbot studied a university community’s lifestyle during the COVID-19 confinement
Cuesta, S. A., Blanes-Selva, V., Portolés, M. G., Conejero, J. A., & Garcia-Gomez, J. M. (2021) · Health Informatics Journal · doi:10.1177/14604582211017944 · Cited by 9 · see citing works
Datafizierung (in) der Bildung
Schiefner-Rohs, M., Hofhues, S., & Breiter, A. (2023) · Digitale Gesellschaft · doi:10.14361/9783839465820 · Cited by 28 · see citing works
Cingillioglu, I. (2024) · Studies in Higher Education · doi:10.1080/03075079.2024.2319870 · Cited by 7 · see citing works
Chatbot utilization in dermatology: a potential amelioration to burnout in dermatology
Diamond, C. E., Rundle, C. W., Albrecht, J. M., & Nicholas, M. W. (2022) · Dermatology Online Journal · doi:10.5070/d328659734 · Cited by 6 · see citing works
Šumak, B., Pušnik, M., Kožuh, I., Šorgo, A., & Brdnik, S. (2025) · Applied Sciences · doi:10.3390/app15020631 · Cited by 5 · see citing works
Bi, T., Buono, R. A., Olugbade, T., Singh, A., Holloway, C., Costanza, E., Williams, A. C. D. C., Gold, N., & Bianchi‐Berthouze, N. (2021) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3461615.3485670 · Cited by 5 · see citing works
Tulsiani, S., Ichimiya, M., Gerard, R., Mills, S., Bingenheimer, J. B., Hair, E. C., Vallone, D., & Evans, W. D. (2022) · JMIR Formative Research · doi:10.2196/37856 · Cited by 5 · see citing works
Chatbot User Experience: Design and Evaluation
Peras, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-76806-4_6 · Cited by 4 · see citing works
Boyer, L., Fernandes, S., Brousse, Y., Llorca, P., Samalin, L., Rahmati, M., Auquier, P., Boussat, B., & Fond, G. B. (2024) · Journal of Epidemiology and Population Health · doi:10.1016/j.jeph.2024.202785 · Cited by 4 · see citing works
Review of Modern Chatbot Technologies: Performance, Capabilities and Applications
Rana, M. R., Bhuiyan, Z. W., Limon, M. H., Rahman, A., Hasan, M., & Habib, M. T. (2025) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro65660.2025.11131741 · Cited by 3 · see citing works
Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis
Meng, H., Yang, Y., Fu, W., Lee, J., Li, Y., & Lee, Y. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3778354 · Cited by 22 · see citing works
Determinants of consumers’ emotions and willingness to use artificial intelligence in Indonesia
Laksmidewi, D., Efendi, & Hoo, W. C. (2024) · Innovative Marketing · doi:10.21511/im.20(4).2024.22 · Cited by 3 · see citing works
Skovfoged, M. M., Mavromati, K., Tvrdá, L., Quinn, T. J., Körner, F., Chiesurin, S., Némcová, V., Mikulík, R., & Knoche, H. O. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3799983 · Cited by 2 · see citing works
Jin, X., & Fan, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3676499 · Cited by 2 · see citing works
Experiences of patients with hard-to-heal wounds: insights from a pilot survey
Wahab, N., & Forsyth, A. (2024) · Journal of Wound Care · doi:10.12968/jowc.2024.0109 · Cited by 2 · see citing works
Contouring a User Centered Chatbot for Diabetes Mellitus
Palanichamy, H. (2022) · International journal of high school research · doi:10.36838/v4i4.16 · Cited by 1 · see citing works
Digitizing the Pre-consultation Experience: Impacts and Design Recommendations
Li, B., Bakar, L. O., Кирик, А. М., Guo, J., Mariakakis, A. T., & Truong, K. N. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791354 · Cited by 1 · see citing works
Assessment and Comparison of Meta-features for Educational Chatbots Data and Survey Data
Oreški, D., Vukovac, D. P., & Hajdin, G. (2022) · WSEAS TRANSACTIONS ON ADVANCES in ENGINEERING EDUCATION · doi:10.37394/232010.2022.19.3 · Cited by 1 · see citing works
Sakakibara, K., Shigemi, D., Toriumi, R., Michihata, N., & Yasunaga, H. (2023) · doi:10.2196/preprints.49611 · Cited by 1 · see citing works
Digitale Methoden in Bildungsforschung und Bildungspraxis
Roeske, A., Büntemeyer, D., Zakharova, I., & Breiter, A. (2023) · Digitale Gesellschaft · doi:10.14361/9783839465820-006
Generation Y Health Care Professionals and Their Acceptance of Chatbots
Zwicky, A., Stallone, V., & Haarmann, J. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-45642-8_23
Zhang, Z., Mendel, T., Yin, W., Raipat, V., & Yabe, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5378600
Ye, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6591685
The rise of conversational agents: can they keep up with humans? Evidence from a meta-analysis
Ebrahimi, S., Farivar, S., & Eshghi, K. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2662405
Chatbot-Administered Surveys in Marketing: A Literature Review
Zhao, Y., Gao, X., & Paas, L. J. (2026) · International Journal of Market Research · doi:10.1177/14707853261445476
“Was This Answer Helpful?”—A Taxonomy for Feedback Mechanisms in Customer Service Chatbots
Schloss, D., Haug, S., & Maedche, A. (2026) · Lecture notes in information systems and organisation · doi:10.1007/978-3-032-08483-5_9
Skovfoged, M. M., Kasl, F., Koščík, M., Loutocký, P., Stupka, V., Mikulík, R., & Knoche, H. O. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816232
Pavão, J., Bastardo, R., Silva, A. G., & Rocha, N. P. (2026) · JMIR Human Factors · doi:10.2196/102911
- Authors
- te Pas, M. E., Rutten, W. G. M. M., Bouwman, R. A., & Buise, M. P.
- Affiliations
- Catharina Ziekenhuis (NL)
- Venue
- JMIR Medical Informatics
- Publisher
- JMIR Publications Inc.
- Published
- 2020-12-07
- Volume
- 8
- Issue
- 12
- Pages
- e21982
- ISSN
- 2291-9694 (ISSN-L)
- PMC
- PMC7752526
- OpenAlex
- W3095503508
- Cited by
- 43
- References
- 28
- Citations by year
- 2026: 8 · 2025: 6 · 2024: 13 · 2023: 7 · 2022: 5 · 2021: 4
- FWCI
- 5.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Informatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- TU/e Research Portal (publishedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- TU/e Research Portal (submittedVersion, other-oa)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- user experience, respondent engagement, response rate, patient feedback, questionnaire completion time
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
