Help! Is my chatbot falling into the uncanny valley? An empirical study of user experience in human-chatbot interaction
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Advances in artificial intelligence strengthen chatbots' ability to resemble human conversational agents.For some application areas, it may be tempting not to be transparent regarding a conversational agent's nature as chatbot or human.However, the uncanny valley theory suggests that such lack in transparency may cause uneasy feelings in the user.In this study, we combined quantitative and qualitative methods to investigate this issue.First, we used a 2 x 2 experimental research design (n = 28) to investigate effects of lack in transparency on the perceived pleasantness of the conversation in addition to perceived human likeness and affinity for the conversational agent.Second, we conducted an exploratory analysis of qualitative participant reports on these conversations.We did not find that a lack in transparency negatively affected user experience, but we identified three factors important to participants' assessments.The findings are of theoretical and practical significance and motivate future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
A Beginner’s Guide to Factor Analysis: Focusing on Exploratory Factor Analysis
Yong, A. G., & Pearce, S. (2013) · Tutorials in Quantitative Methods for Psychology · doi:10.20982/tqmp.09.2.p079 · Cited by 3,304
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 1×
“In a study of human interaction with computers, Nass, Steuer, and Tauber (1994) showed that people tended to demonstrate responses that adhered to social norms.”
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176 · cited 3×
“For instance, Luger and Sellen (2016), in an interview study of Siri, Cortana, and Google Now users, identified substantial mismatch between user expectations and system capabilities and, consequently, argued for the need for conversational systems to be open about their limitations.”
BackgroundOnline Human-Bot Interactions: Detection, Estimation, and Characterization
Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14871 · Cited by 981 · cited 1×
“In recent years, the distinction between human and automated conversational agents in social media services such as Twitter has become blurred, where a large proportion of automated accounts pose as human conversational agents (Varol, Ferrara, Davis, Menczer, & Flammini, 2017).”
BackgroundWalther, J. B. (2007) · Computers in Human Behavior · doi:10.1016/j.chb.2006.05.002 · Cited by 915 · cited 6×
“Although the term chatbot is relatively recent, software agents communicating with users in natural language have been studied since Weizenbaum’s ELIZA (1966). Such agents come by various terms, such as conversational agents, chatterbots, and chatbots. In a study of human interaction with computers, Nass, Steuer, and…”
BackgroundInfluentialHill, J., Randolph Ford, W., & Farreras, I. G. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.02.026 · Cited by 856 · cited 4×
“Even though users tend to use shorter sentences in their interaction with chatbots, research indicates that they send more messages to the chatbot compared to a human, which might be an indication of users’ motivation for interacting with chatbots (Hill et al., 2015), as has also been found in other studies (e.g., Cru…”
BackgroundResultInfluentialBrandtzaeg, P. B., & Følstad, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-70284-1_30 · Cited by 788 · cited 1×
“…how chatbots have an advantage due to their ability to provide fast answers, which makes them fit for working within customer service, as studies have shown that one of the key motivations for utilizing chatbots is productivity and saving time (Brandtzaeg & Følstad, 2017; Luger & Sellen, 2016).”
BackgroundSelf-citationCrutzen, R., Peters, G.-J. Y., Portugal, S. D., Fisser, E. M., & Grolleman, J. J. (2011) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2010.09.002 · Cited by 205 · cited 1×
“…chatbots, research indicates that they send more messages to the chatbot compared to a human, which might be an indication of users’ motivation for interacting with chatbots (Hill et al., 2015), as has also been found in other studies (e.g., Crutzen, Peters, Portugal, Fisser, and Grolleman, 2011).”
ResultIn search of the uncanny valley
Pollick, F. E. (2010) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-642-12630-7_8 · Cited by 91 · cited 1×
“The uncanny valley effect has been argued to be one of the reasons why movies such as Final Fantasy have failed (Pollick, 2010).”
Background
Cited by
Chatbots: History, technology, and applications
Adamopoulou, E., & Moussiades, L. (2020) · Machine Learning with Applications · doi:10.1016/j.mlwa.2020.100006 · Cited by 1,090
Rapp, A., Curti, L., & Boldi, A. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102630 · Cited by 643
Mariani, M. M., Hashemi, N., & Wirtz, J. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.113838 · Cited by 343
Chandra, S., Shirish, A., & Srivastava, S. C. (2022) · Journal of Management Information Systems · doi:10.1080/07421222.2022.2127441 · Cited by 315
Nicolescu, L., & Tudorache, M. T. (2022) · Electronics · doi:10.3390/electronics11101579 · Cited by 280
Metadata
- Authors
- Skjuve, M., Haugstveit, I. M., Følstad, A., & Brandtzaeg, P. B.
- Venue
- Human Technology
- Publisher
- Centre of Sociological Research, NGO
- Published
- 2019-02-28
- Pages
- 30-54
- ISSN
- 1795-6889 (ISSN-L)
- OpenAlex
- W2920375850
- Cited by
- 138
- References
- 10
- FWCI
- 4.95
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Help! Is my chatbot falling into the uncanny valley? An empirical study of user experience in human-chatbot interaction. https://app.cochat.ai/papers/10.17011/ht/urn.201902201607
- The paper
- Skjuve, M., Haugstveit, I. M., Følstad, A., & Brandtzaeg, P. B. (2019). Help! Is my chatbot falling into the uncanny valley? An empirical study of user experience in human-chatbot interaction. Human Technology. https://doi.org/10.17011/ht/urn.201902201607
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.