The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions
Open accessAbstract
The rise of increasingly more powerful chatbots offers a new way to collect information through conversational surveys, where a chatbot asks open-ended questions, interprets a user’s free-text responses, and probes answers whenever needed. To investigate the effectiveness and limitations of such a chatbot in conducting surveys, we conducted a field study involving about 600 participants. In this study with mostly open-ended questions, half of the participants took a typical online survey on Qualtrics and the other half interacted with an AI-powered chatbot to complete a conversational survey. Our detailed analysis of over 5,200 free-text responses revealed that the chatbot drove a significantly higher level of participant engagement and elicited significantly better quality responses measured by Gricean Maxims in terms of their informativeness, relevance, specificity, and clarity. Based on our results, we discuss design implications for creating AI-powered chatbots to conduct effective surveys and beyond.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 1×
“In addition to examining user satisfaction and usability of conversational AI agents, HCI researchers have investigated how agent behavior impacts users’ social perceptions, such as trust [15], rapport [9, 58], anthropomorphism, and likability [5].”
BackgroundAGE DIFFERENCES IN TECHNOLOGY ADOPTION DECISIONS: IMPLICATIONS FOR A CHANGING WORK FORCE
MORRIS, M. G., & VENKATESH, V. (2000) · Personnel Psychology · doi:10.1111/j.1744-6570.2000.tb00206.x · Cited by 1,723 · cited 1×
“Since research suggests that demographics influence people’s behavior with new technology [56, 75], all analyses was controlled for participants’ age, gender, and education.”
BackgroundPsychological Predictors of Problem Mobile Phone Use
Bianchi, A., & Phillips, J. G. (2005) · CyberPsychology & Behavior · doi:10.1089/cpb.2005.8.39 · Cited by 1,514
Venkatesh, V., Morris, M. G., & Ackerman, P. L. (2000) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.2000.2896 · Cited by 1,237
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
It's only a computer: virtual humans increase willingness to disclose
Lucas, G. M., Gratch, J., King, A., & Morency, L.-P. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.04.043 · Cited by 977
McTear, M., Callejas, Z., & Griol, D. (2016) · doi:10.1007/978-3-319-32967-3 · Cited by 594
Relational agents: a model and implementation of building user trust
Bickmore, T., & Cassell, J. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365304 · Cited by 479
Sproull, L., Subramani, M., Kiesler, S., Walker, J., & Waters, K. (1996) · Human-Computer Interaction · doi:10.1207/s15327051hci1102_1 · Cited by 399
Machine Heuristic: When We Trust Computers More than Humans with Our Personal Information
Sundar, S. S., & Kim, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300768 · Cited by 347
Research works printed but not linked: 74
- David A. Aaker . 1976 . Design and analysis: A researcher’s handbook . Journal of Marketing Research 13 , 000003 (1976), 318 . David A. Aaker. 1976. Design and analysis: A researcher’s handbook. Journal of Marketing Research 13, 000003 (1976), 318.
- James F. Allen , Donna K. Byron , Myroslava Dzikovska , George Ferguson , Lucian Galescu , and Amanda Stent . 2001 . Toward conversational human-computer interaction . AI Magazine 22 , 4 (2001), 27 . James F. Allen, Donna K. Byron, Myroslava Dzikovska, George Ferguson, Lucian Galescu, and Amanda Stent. 2001. Toward conversational human-computer interaction. AI Magazine 22, 4 (2001), 27.
- Irwin Altman and Dalmas A . Taylor . 1973 . Social Penetration : The Development of Interpersonal Relationships. Holt, Rinehart 8 Winston . Irwin Altman and Dalmas A. Taylor. 1973. Social Penetration: The Development of Interpersonal Relationships. Holt, Rinehart 8 Winston.
- 10.1093/poq/nfw088 · DOI
- C. Bauer , K. Figl , and R. Motschnig-Pitrik . 2010 . Introducing ‘active listening’ to instant messaging and e-mail: Benefits and limitations . IADIS International Journal on WWW/Internet 7 , 2 (2010), 1 -- 17 . C. Bauer, K. Figl, and R. Motschnig-Pitrik. 2010. Introducing ‘active listening’ to instant messaging and e-mail: Benefits and limitations. IADIS International Journal on WWW/Internet 7, 2 (2010), 1--17.
- 10.1177/0894439311435305 · DOI
- Pazit Ben-Nun . 2008 . Respondent fatigue . Encyclopedia of Survey Research Methods 2 (2008), 742 -- 743 . Pazit Ben-Nun. 2008. Respondent fatigue. Encyclopedia of Survey Research Methods 2 (2008), 742--743.
- 10.1007/978-3-642-23974-8_7 · DOI
- Dan Bohus and Alexander I . Rudnicky . 2009 . The ravenclaw dialog management framework: Architecture and systems. Computer Speech 8 Language 23, 3 (2009), 332--361. Dan Bohus and Alexander I. Rudnicky. 2009. The ravenclaw dialog management framework: Architecture and systems. Computer Speech 8 Language 23, 3 (2009), 332--361.
- 10.1007/BF02763643 · DOI
- 10.1145/355112.355123 · DOI · Paper page
- Frederick G. Conrad , Mick P. Couper , Roger Tourangeau , Mirta Galesic , and T. Yan . 2005. Interactive feedback can improve the quality of responses in web surveys . In Proceedings of the Section on Survey Research Methods. 3835--3840 . Frederick G. Conrad, Mick P. Couper, Roger Tourangeau, Mirta Galesic, and T. Yan. 2005. Interactive feedback can improve the quality of responses in web surveys. In Proceedings of the Section on Survey Research Methods. 3835--3840.
- 10.18653/v1/P17-4017 · DOI · Paper page
- 10.1177/0894439315573744 · DOI
- David DeVault , Ron Artstein , Grace Benn , Teresa Dey , Ed Fast , Alesia Gainer , Kallirroi Georgila , Jon Gratch , Arno Hartholt , Margaux Lhommet , 2014 . SimSensei kiosk: A virtual human interviewer for healthcare decision support . In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 1061--1068 . David DeVault, Ron Artstein, Grace Benn, Teresa Dey, Ed Fast, Alesia Gainer, Kallirroi Georgila, Jon Gratch, Arno Hartholt, Margaux Lhommet, et al. 2014. SimSensei kiosk: A virtual human interviewer for healthcare decision support. In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 1061--1068.
- 10.3115/992628.992686 · DOI
- 10.1016/j.specom.2004.02.001 · DOI
- S. Egelman E. Chi and S. Dow. 2014. Crowdsourcing in HCI research. In Ways of Knowing in HCI J. Olson and W. Kellogg (Eds.). Springer 267--289. S. Egelman E. Chi and S. Dow. 2014. Crowdsourcing in HCI research. In Ways of Knowing in HCI J. Olson and W. Kellogg (Eds.). Springer 267--289. · DOI
- 10.1108/10662240510590360 · DOI · Paper page
- 10.1016/j.chb.2018.02.025 · DOI
- Eric N. Forsythand and Craig H. Martell. 2007. Lexical and discourse analysis of online chat dialog . In Proceedings of the International Conference on Semantic Computing. IEEE, 19--26 . Eric N. Forsythand and Craig H. Martell. 2007. Lexical and discourse analysis of online chat dialog. In Proceedings of the International Conference on Semantic Computing. IEEE, 19--26.
- 10.1177/152582202237725 · DOI · Paper page
- 10.1145/2702123.2702443 · DOI · Paper page
- 10.1109/TE.2005.856149 · DOI · Paper page
- Herbert P. Grice . 1975. Logic and conversation . In Speech Acts . Brill , 41–58. Herbert P. Grice. 1975. Logic and conversation. In Speech Acts. Brill, 41–58.
- Elisabeth Hayes and Maryellen Ohrnberger . 2013 . The gamer generation teaches school: The gaming practices and attitudes towards technology of pre-service teachers . Journal of Technology and Teacher Education 21 , 2 (2013), 154 -- 177 . Elisabeth Hayes and Maryellen Ohrnberger. 2013. The gamer generation teaches school: The gaming practices and attitudes towards technology of pre-service teachers. Journal of Technology and Teacher Education 21, 2 (2013), 154--177.
- 10.1093/ijpor/edl028 · DOI
- Charles T. Hemphill , John J. Godfrey , and George R. Doddington . 1990. The ATIS spoken language systems pilot corpus . In Proceedings of the workshop on Speech and Natural Language. Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990. The ATIS spoken language systems pilot corpus. In Proceedings of the workshop on Speech and Natural Language.
- 10.1177/0956797611436349 · DOI · Paper page
- Knut Hofland and Stig Johansson . 1982. Word Frequencies in British and American English . Norwegian Computing Centre for the Humanities. Knut Hofland and Stig Johansson. 1982. Word Frequencies in British and American English. Norwegian Computing Centre for the Humanities.
- 10.1017/s1351324900002497 · DOI · Paper page
- 10.1145/2736277.2741669 · DOI
- Douglas Samuel Jones . 1979. Elementary Information Theory . Clarendon Press . Douglas Samuel Jones. 1979. Elementary Information Theory. Clarendon Press.
- Jie Kang , Kyle Condiff , Shuo Chang , Joseph A. Konstan , Loren Terveen , and F. Maxwell Harper . 2017. Understanding how people use natural language to ask for recommendations . In Proceedings of the 11th ACM Conference on Recommender Systems. ACM, 229--237 . Jie Kang, Kyle Condiff, Shuo Chang, Joseph A. Konstan, Loren Terveen, and F. Maxwell Harper. 2017. Understanding how people use natural language to ask for recommendations. In Proceedings of the 11th ACM Conference on Recommender Systems. ACM, 229--237.
- 10.1145/3290605.3300316 · DOI · Paper page
- 10.1146/annurev.psych.50.1.537 · DOI · Paper page
- Paul J. Lavrakas . 2008. Encyclopedia of Survey Research Methods . Sage Publications . Paul J. Lavrakas. 2008. Encyclopedia of Survey Research Methods. Sage Publications.
- 10.3389/fpsyg.2016.01771 · DOI
- 10.1145/3025171.3025206 · DOI
- Vera Q. Liao , Muhammed Masud Hussain , Praveen Chandar , Matthew Davis , Marco Crasso , Dakuo Wang , Michael Muller , Sadat N. Shami , and Werner Geyer . 2018 . All work and no play? conversations with a question-and-answer chatbot in the wild . In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18) . ACM, New York, NY , Vol. 13 . Vera Q. Liao, Muhammed Masud Hussain, Praveen Chandar, Matthew Davis, Marco Crasso, Dakuo Wang, Michael Muller, Sadat N. Shami, and Werner Geyer. 2018. All work and no play? conversations with a question-and-answer chatbot in the wild. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY, Vol. 13.
- Joseph Carl Robnett Licklider . 1960. Man-computer symbiosis. IRE Transactions on Human Factors in Electronics1 ( 1960 ), 4--11. Joseph Carl Robnett Licklider. 1960. Man-computer symbiosis. IRE Transactions on Human Factors in Electronics1 (1960), 4--11.
- 10.18653/v1/D16-1230 · DOI · Paper page
- S. Louw , R. W. Todd , and P. Jimakorn . 2010. Active listening in qualitative research interviews . In Proceedings of the Doing Research in Applied Linguistics. 71--82 . S. Louw, R. W. Todd, and P. Jimakorn. 2010. Active listening in qualitative research interviews. In Proceedings of the Doing Research in Applied Linguistics. 71--82.
- 10.1177/0894439313485201 · DOI · Paper page
- 10.3115/974499.974549 · doi:10.3115/974499.974549
- Hendrik Muller , Aaron Sedley , and Elizabeth Ferrall-Nunge . 2014. Survey research in HCI . In Ways of Knowing in HCI . Springer , 229--266. Hendrik Muller, Aaron Sedley, and Elizabeth Ferrall-Nunge. 2014. Survey research in HCI. In Ways of Knowing in HCI. Springer, 229--266.
- 10.1007/978-3-319-07230-2_45 · DOI
- Marije Oudejans and Leah Melani Christian. 2010. Using interactive features to motivate and probe responses to open-ended questions. In Social and Behavioral Research and the Internet. 215--244. Marije Oudejans and Leah Melani Christian. 2010. Using interactive features to motivate and probe responses to open-ended questions. In Social and Behavioral Research and the Internet. 215--244. · doi:10.4324/9780203844922-9
- W. K. Park. 2005. Mobile phone addiction. In Mobile Communications. 253–272. W. K. Park. 2005. Mobile phone addiction. In Mobile Communications. 253–272. · DOI · Paper page
- Richard Paul and Linda Elder . 2006. The Thinker’s Guide to the Art of Socratic Questioning . Foundation for Critical Thinking Dillon Beach , CA. Richard Paul and Linda Elder. 2006. The Thinker’s Guide to the Art of Socratic Questioning. Foundation for Critical Thinking Dillon Beach, CA.
- 10.1037/0022-3514.41.5.847 · doi:10.1037/0022-3514.41.5.847 · Paper page
- 10.1002/ir.97 · DOI · Paper page
- 10.1002/ir.101 · DOI · Paper page
- 10.1145/3020165.3020183 · DOI · Paper page
- Ursa Reja , Katja Lozar Manfreda , Valentina Hlebec , and Vasja Vehovar . 2003 . Open-ended vs. close-ended questions in web questionnaires . Developments in Applied Statistics 19 , 1 (2003), 159 -- 177 . Ursa Reja, Katja Lozar Manfreda, Valentina Hlebec, and Vasja Vehovar. 2003. Open-ended vs. close-ended questions in web questionnaires. Developments in Applied Statistics 19, 1 (2003), 159--177.
- Francesco Ricci , Lior Rokach , and Bracha Shapira . 2015. Recommender systems: Introduction and challenges . In Recommender Systems Handbook . Springer , 1--34. Francesco Ricci, Lior Rokach, and Bracha Shapira. 2015. Recommender systems: Introduction and challenges. In Recommender Systems Handbook. Springer, 1--34.
- P. H. Rossi J. D. Wright and A. B. Anderson (Eds.). 2013. Handbook of Survey Research. Academic Press. P. H. Rossi J. D. Wright and A. B. Anderson (Eds.). 2013. Handbook of Survey Research. Academic Press.
- Shruti Sannon , Brett Stoll , Dominic DiFranzo , Malte Jung , and Natalya N. Bazarova . 2018. How personification and interactivity influence stress-related disclosures to conversational agents . In Proceedings of the Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 285--288 . Shruti Sannon, Brett Stoll, Dominic DiFranzo, Malte Jung, and Natalya N. Bazarova. 2018. How personification and interactivity influence stress-related disclosures to conversational agents. In Proceedings of the Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 285--288.
- Jessica Schroeder , Chelsey Wilkes , Kael Rowan , Arturo Toledo , Ann Paradiso , Mary Czerwinski , Gloria Mark , and Marsha M. Linehan . 2018. Pocket skills: A conversational mobile web app to support dialectical behavioral therapy . In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 398 . Jessica Schroeder, Chelsey Wilkes, Kael Rowan, Arturo Toledo, Ann Paradiso, Mary Czerwinski, Gloria Mark, and Marsha M. Linehan. 2018. Pocket skills: A conversational mobile web app to support dialectical behavioral therapy. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 398.
- 10.1145/1541948.1541983 · DOI
- 10.1145/3173574.3173965 · DOI
- Eleanor Singer and Mick P. Couper . 2017 . Some methodological uses of responses to open questions and other verbatim comments in quantitative surveys. Methods, Data, Analyses : A Journal for Quantitative Methods and Survey Methodology 11 , 2 (2017), 115 -- 134 . Eleanor Singer and Mick P. Couper. 2017. Some methodological uses of responses to open questions and other verbatim comments in quantitative surveys. Methods, Data, Analyses: A Journal for Quantitative Methods and Survey Methodology 11, 2 (2017), 115--134.
- 10.1145/3173574.3174178 · DOI
- 10.1145/3176349.3176388 · DOI
- 10.1016/j.artint.2007.09.009 · DOI · Paper page
- David Traum . 2017. Computational approaches to dialogue . In The Routledge Handbook of Language and Dialogue, Edda Weigand (Ed.). Routledge , 143–161. David Traum. 2017. Computational approaches to dialogue. In The Routledge Handbook of Language and Dialogue, Edda Weigand (Ed.). Routledge, 143–161.
- 10.1145/3176349.3176387 · DOI
- Marilyn A. Walker , Diane J. Litman , Candace A. Kamm , and Alicia Abella . 1997 . PARADISE: A framework for evaluating spoken dialogue agents . In Proceedings of the 8th Conference of the European Chapter of the Association for Computational Linguistics. Marilyn A. Walker, Diane J. Litman, Candace A. Kamm, and Alicia Abella. 1997. PARADISE: A framework for evaluating spoken dialogue agents. In Proceedings of the 8th Conference of the European Chapter of the Association for Computational Linguistics.
- 10.1145/2702123.2702293 · DOI
- 10.1177/0894439313505829 · DOI
- 10.1145/3173574.3173662 · DOI
- 10.1145/3301275.3302264 · DOI
- 10.1145/3025453.3025496 · DOI · Paper page
- 10.1145/3172944.3172998 · doi:10.1145/3172944.3172998
Cited by
Chen, Y., Jensen, S., Albert, L. J., Gupta, S., & Lee, T. (2023) · Information Systems Frontiers · doi:10.1007/s10796-022-10291-4 · Cited by 685
The impact of ChatGPT on higher education
Dempere, J., Modugu, K., Hesham, A., & Ramasamy, L. K. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1206936 · Cited by 519
Yang, Z., Xu, X., Yao, B., Rogers, E., Zhang, S., Intille, S., Shara, N., Gao, G. G., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659625 · Cited by 120
Exploring and Promoting Diagnostic Transparency and Explainability in Online Symptom Checkers
Tsai, C.-H., You, Y., Gui, X., Kou, Y., & Carroll, J. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445101 · Cited by 86
Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
Wei, J., Kim, S., Jung, H., & Kim, Y.-H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637364 · Cited by 84
Metadata
- Authors
- Xiao, Z., Zhou, M. X., Liao, Q. V., Mark, G., Chi, C., Chen, W., & Yang, H.
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2020-06-13
- Volume
- 27
- Issue
- 3
- Pages
- 1-37
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W4288080268
- Cited by
- 185
- References
- 15
- FWCI
- 23.47
- Type
- journal article
- Language
- en
- Open access
- bronze
- Funders
- Air Force Office of Scientific Research (FA9550-15-C-0032)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions. https://app.cochat.ai/papers/10.1145/3381804
- The paper
- Xiao, Z., Zhou, M. X., Liao, Q. V., Mark, G., Chi, C., Chen, W., & Yang, H. (2020). Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions. ACM Transactions on Computer-Human Interaction. https://doi.org/10.1145/3381804
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.