The Chatbot Usability Scale: the Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents
Open accessThe DOI matches its Crossref record; 67 of the 68 references it cites were found in a registry or an index.
Abstract
Standardised tools to assess a user’s satisfaction with the experience of using chatbots and conversational agents are currently unavailable. This work describes four studies, including a systematic literature review, with an overall sample of 141 participants in the survey (experts and novices), focus group sessions and testing of chatbots to (i) define attributes to assess the quality of interaction with chatbots and (ii) the designing and piloting a new scale to measure satisfaction after the experience with chatbots. Two instruments were developed: (i) A diagnostic tool in the form of a checklist (BOT-Check). This tool is a development of previous works which can be used reliably to check the quality of a chatbots experience in line with commonplace principles. (ii) A 15-item questionnaire (BOT Usability Scale, BUS-15) with estimated reliability between .76 and .87 distributed in five factors. BUS-15 strongly correlates with UMUX-LITE by enabling designers to consider a broader range of aspects usually not considered in satisfaction tools for non-conversational agents, e.g. conversational efficiency and accessibility, quality of the chatbot’s functionality and so on. Despite the convincing psychometric properties, BUS-15 requires further testing and validation. Designers can use it as a tool to assess products, thus building independent databases for future evaluation of its reliability, validity and sensitivity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
SUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
Usability Evaluation In Industry
(1996) · CRC Press eBooks · doi:10.1201/9781498710411 · Cited by 4,354
Answering the Call for a Standard Reliability Measure for Coding Data
Hayes, A. F., & Krippendorff, K. (2007) · Communication Methods and Measures · doi:10.1080/19312450709336664 · Cited by 4,225 · cited 2×
“The agreement among the participants was equal to 0.780, which is acceptably higher than the minimum level of .667 for the Krippendorff ’ s Alpha [35].”
MethodResultInfluentialFitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730
The measurement of user information satisfaction
Ives, B., Olson, M. H., & Baroudi, J. J. (1983) · Communications of the ACM · doi:10.1145/358413.358430 · Cited by 1,820
Service loyalty: The effects of service quality and the mediating role of customer satisfaction
Caruana, A. (2002) · European Journal of Marketing · doi:10.1108/03090560210430818 · Cited by 1,528
Trust in a specific technology: An investigation of its components and measures
Mcknight, D. H., Carter, M., Thatcher, J. B., & Clay, P. F. (2011) · ACM Transactions on Management Information Systems · doi:10.1145/1985347.1985353 · Cited by 1,185
Go, E., & Sundar, S. S. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.01.020 · Cited by 1,089
Research works printed but not linked: 13
- Charlton JI (2000) Nothing about us without us: Disability oppression and empowerment. Univ of California Press
- DeVellis, R. F. (2016). Scale development: Theory and applications (Vol. 26). Sage publications.
- Dillon A (2001) Beyond Usability: Process, Outcome and Affect in human computer interactions. Canadian Journal of Information and Library Science 26(4)
- Feine, J., Morana, S., & Gnewuch, U. (2019). Measuring service encounter satisfaction with customer service chatbots using sentiment analysis 14th International Conference on Wirtschaftsinformatik (WI2019), Siegen, Germany. https://aisel.aisnet.org/wi2019/
- Harkous, H., Fawaz, K., Shin, K. G., & Aberer, K. (2016). Pribots: Conversational privacy with chatbots 12th Symposium on Usable Privacy and Security, Denver, Colorado, US.
- Lewis, J. R. (2001). Psychometric properties of the mean opinion scale. Proceedings of HCI International 2001: Usability Evaluation and Interface Design, 149-153.
- Munteanu, C., & Boldea, M. (2000). MDWOZ: A Wizard of Oz Environment for Dialog Systems Development International Conference on Language Resources and Evaluation (LREC’00), Athens, Greece.
- Peras, D. (2018). Chatbot evaluation metrics. Economic and Social Development: Book of Proceedings, 89-97
- Radziwill, N. M., & Benton, M. C. (2017). Evaluating quality of chatbots and intelligent conversational agents. arXiv preprint arXiv:1704.04579. · arXiv
- Revelle W (2011) An overview of the psych package. Department of Psychology Northwestern University. Accessed on March 3(2012):1–25
- Saenz, J., Burgess, W., Gustitis, E., Mena, A., & Sasangohar, F. (2017). The usability analysis of chatbot technologies for internal personnel communications Industrial and Systems Engineering Conference Pittsburgh, Pennsylvania, US.
- Tay L, Jebb A (2017) Scale Creation. In: Rogelberg SG (ed) The SAGE encyclopedia of industrial and organizational psychology, 2nd edn. SAGE Publications, Inc.
- Wilson HJ, Daugherty P, Bianzino N (2017) The jobs that artificial intelligence will create. MIT Sloan Management Review 58(4):14
Cited by
Artificial intelligence applications in Latin American higher education: a systematic review
Salas-Pilco, S. Z., & Yang, Y. (2022) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-022-00326-w · Cited by 330
“These applications are being integrated in social networks, websites, and a variety of platforms (Borsci et al., 2021).”
BackgroundAl Naqbi, H., Bahroun, Z., & Ahmed, V. (2024) · Sustainability · doi:10.3390/su16031166 · Cited by 302
Nicolescu, L., & Tudorache, M. T. (2022) · Electronics · doi:10.3390/electronics11101579 · Cited by 280
Kim, J., & Im, I. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107512 · Cited by 220
Shahzad, M. F., Xu, S., An, X., & Javed, I. (2024) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2024.103867 · Cited by 202
Sharma, K., Nguyen, A., & Hong, Y. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13459 · Cited by 196
Evaluating privacy, security, and trust perceptions in conversational AI: A systematic review
Leschanowsky, A., Rech, S., Popp, B., & Bäckström, T. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108344 · Cited by 178
AI Companions Reduce Loneliness
De Freitas, J., Oğuz-Uğuralp, Z., Uğuralp, A. K., & Puntoni, S. (2026) · Journal of Consumer Research · doi:10.1093/jcr/ucaf040 · Cited by 165
Lai, W. Y. W., & Lee, J. S. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100291 · Cited by 141
Ha, J., Jeon, H., Han, D. S., Seo, J., & Oh, C. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642472 · Cited by 135
Papneja, H., & Yadav, N. (2024) · Personal and Ubiquitous Computing · doi:10.1007/s00779-024-01823-7 · Cited by 114
Proactive Conversational Agents with Inner Thoughts
Liu, X., Fang, S., Shi, W., Wu, C.-S., Igarashi, T., & Chen, X. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713760 · Cited by 114
Irfan, B., Kuoppamäki, S., Hosseini, A., & Skantze, G. (2025) · Autonomous Robots · doi:10.1007/s10514-025-10190-y · Cited by 108
Aslam, U. (2023) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2023.103377 · Cited by 95
Nie, J., Shao, H., Fan, Y., Shao, Q., You, H., Preindl, M., & Jiang, X. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3712299 · Cited by 87
Jo, H. (2025) · Information Development · doi:10.1177/02666669231202764 · Cited by 84
AI Companions Reduce Loneliness
De Freitas, J., Uğuralp, A. K., Uğuralp, Z., & Puntoni, S. (2024) · doi:10.2139/ssrn.4893097 · Cited by 80
A confirmatory factorial analysis of the Chatbot Usability Scale: a multilanguage validation
Borsci, S., Schmettow, M., Malizia, A., Chamberlain, A., & van der Velde, F. (2022) · Personal and Ubiquitous Computing · doi:10.1007/s00779-022-01690-0 · Cited by 76
Self-citationKayalı, B., Yavuz, M., Balat, Ş., & Çalışan, M. (2023) · Australasian Journal of Educational Technology · doi:10.14742/ajet.8915 · Cited by 75
Conversational AI Chatbots in library research: An integrative review and future research agenda
Aboelmaged, M. G. S., Bani‐Melhem, S., Al-Hawari, M. A., & Ahmad, I. (2024) · Journal of Librarianship and Information Science · doi:10.1177/09610006231224440 · Cited by 71
Development and validation of the perceived interactivity of learner-AI interaction scale
Wang, F., Cheung, A., Chai, C. S., & Liu, J. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12963-x · Cited by 66
Believe in artificial intelligence? A user study on the chatGPT’s fake information impact.
Amaro, I., Barra, P., Greca, A. D., Francese, R., & Tucci, C. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3291539 · Cited by 60
Ngo, T. T. A., Tran, T. T., An, G. K., & Nguyen, P. T. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3383773 · Cited by 56
Chatbots for Smoking Cessation: Scoping Review
Whittaker, R., Dobson, R., & Garner, K. (2022) · Journal of Medical Internet Research · doi:10.2196/35556 · Cited by 55
Cheah, M. H., Gan, Y. N., Altice, F. L., Wickersham, J. A., Shrestha, R., Salleh, N. A. M., Ng, K. S., Azwa, I., Balakrishnan, V., Kamarulzaman, A., et al. (2024) · JMIR Human Factors · doi:10.2196/52055 · Cited by 53
“The acceptability of the chatbot was measured using the standardized System Usability Scale [23] and an adjusted Chatbot Usability Scale [24].”
MethodLi, Z. S., Arony, N. N., Awon, A. M., Damian, D., & Xu, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.17325 · Cited by 49
Alfirević, N., Praničević, D. G., & Mabić, M. (2024) · Sustainability · doi:10.3390/su16124929 · Cited by 48
Martinengo, L., Lin, X., Jabir, A. I., Kowatsch, T., Atun, R. A., Car, J., & Car, L. T. (2023) · Journal of Medical Internet Research · doi:10.2196/50767 · Cited by 45
CRS-Que : A User-centric Evaluation Framework for Conversational Recommender Systems
Jin, Y., Chen, L., Cai, W., & Zhao, X. (2024) · ACM Transactions on Recommender Systems · doi:10.1145/3631534 · Cited by 45
Chatbot for communicating with university students in emergency situation
Balderas, A., Mena, R. F. G., Huerta, M., Mora, N., & Dodero, J. M. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e19517 · Cited by 45
Jones, G. M., Satran, S., & Satyanarayan, A. (2025) · Big Data & Society · doi:10.1177/20539517241303118 · Cited by 41
Satyanarayan, A., & Jones, G. M. (2024) · doi:10.21428/e4baedd9.2d7598a2 · Cited by 40
Fitrianie, S., Bruijnes, M., Abdulrahman, A., & Brinkman, W. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103482 · Cited by 38
Arora, S., Rajesh, A., Misra, R., & Singh, G. (2025) · Management Decision · doi:10.1108/md-01-2025-0093 · Cited by 38
Borsci, S., & Schmettow, M. (2024) · Personal and Ubiquitous Computing · doi:10.1007/s00779-024-01834-4 · Cited by 36
Self-citationNisha Pradeepa, S. P., Asokk, D., Prasanna, S., & Alam, A. S. (2024) · Journal of Systems and Information Technology · doi:10.1108/jsit-04-2023-0062 · Cited by 34
A Review of Subjective Scales Measuring the User Experience of Voice Assistants
Faruk, L. I. D., Babakerkhell, M. D., Mongkolnam, P., Chongsuphajaisiddhi, V., Funilkul, S., & Pal, D. (2024) · IEEE Access · doi:10.1109/access.2024.3358423 · Cited by 32
Usability testing and trust analysis of a mental health and wellbeing chatbot
Boyd, K., Potts, C., Bond, R., Mulvenna, M. D., Broderick, T., Burns, C., Bickerdike, A., McTear, M., Kostenius, C., Vakaloudis, A., et al. (2022) · European Conference on Cognitive Ergonomics · doi:10.1145/3552327.3552348 · Cited by 31
Understanding the Determinants of Using Government AI-Chatbots by Citizens in Saudi Arabia
Abed, S. S. (2024) · International Journal of Electronic Government Research · doi:10.4018/ijegr.349733 · Cited by 30
Where does AI play a major role in the new product development and product management process?
Witkowski, A., & Wodecki, A. (2025) · Management Review Quarterly · doi:10.1007/s11301-025-00533-5 · Cited by 30
Exploring LLM-powered multi-session human-robot interactions with university students
Mauliana, M., Ashok, A., Czernochowski, D., & Berns, K. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1585589 · Cited by 30
A Survey of Personality, Persona, and Profile in Conversational Agents and ChatbotsPreprint
Sutcliffe, R. F. E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.00609 · Cited by 30
Alabbas, A., & Alomar, K. (2025) · Future Internet · doi:10.3390/fi17020064 · Cited by 29
Marconi, L., Longo, L., & Cabitza, F. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8020028 · Cited by 29
Jeon, S., Lee, S., Kim, E. H., Eun, J., Lee, K., Lim, H., & Lee, J. (2025) · JMIR Formative Research · doi:10.2196/72553 · Cited by 28
Kuoppamäki, S., Jaberibraheem, R., Hellstrand, M., & McMillan, D. (2023) · International Journal of Social Robotics · doi:10.1007/s12369-023-01055-4 · Cited by 27
Huang, Y., Chen, H., & Hu, C. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1700117 · Cited by 26
Not in our image: rethinking anthropomorphism in expert chatbot design
Karimova, G. Z. (2025) · AI & Society · doi:10.1007/s00146-025-02438-z · Cited by 25
Framework for Guiding the Development of High-Quality Conversational Agents in Healthcare
Denecke, K. (2023) · Healthcare · doi:10.3390/healthcare11081061 · Cited by 25
Marcinkevage, C. H., & Kumar, A. (2025) · Computers · doi:10.3390/computers14030101 · Cited by 24
Ho, H. P., Ramesh, V., Zaloudek, I., Rikhtehgar, D. J., & Wang, S. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712145 · Cited by 24
Leschanowsky, A., Popp, B., & Peters, N. (2023) · European Symposium on Usable Security (EuroUSEC) · doi:10.1145/3617072.3617106 · Cited by 24
Sonntag, M., Mehmann, J., & Teuteberg, F. (2023) · Journal of Organizational Computing and Electronic Commerce · doi:10.1080/10919392.2023.2276631 · Cited by 24
Ciao AI: the Italian adaptation and validation of the Chatbot Usability Scale
Borsci, S., Prati, E., Malizia, A., Schmettow, M., Chamberlain, A., & Federici, S. (2023) · Personal and Ubiquitous Computing · doi:10.1007/s00779-023-01731-2 · Cited by 23
Self-citationAutomotive fault nowcasting with machine learning and natural language processing
Pavlopoulos, J., Romell, A., Curman, J., Steinert, O., Lindgren, T., Borg, M., & Randl, K. (2023) · Machine Learning · doi:10.1007/s10994-023-06398-7 · Cited by 23
Sun, X., de Wit, J., Li, Z., Pei, J., Ali, A. E., & Bosch, J. A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757655 · Cited by 22
Understanding anthropomorphic voice-AI chatbot continuance from a human-AI interaction perspective
Xie, W., Yang, S., Li, Y., & Zhou, S. (2025) · Behaviour & Information Technology · doi:10.1080/0144929x.2024.2427107 · Cited by 21
Chatbots in Pharmacy: A Boon or a Bane for Patient Care and Pharmacy Practice?
Ramadhani, C. (2023) · Sciences of Pharmacy · doi:10.58920/sciphar02030001 · Cited by 21
Conversational Interfaces in IoT Ecosystems: Where We Are, What Is Still Missing
Gallo, S., Paternò, F., & Malizia, A. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627775 · Cited by 21
Self-citationLatif, M. S., Wang, J., Shahzad, M., & Mursil, M. (2024) · Internet Research · doi:10.1108/intr-08-2023-0666 · Cited by 21
Liu, Y., & Martens, J.-B. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103227 · Cited by 20
Ziegfeld, L., Scala, D., & Cremers, A. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101696 · Cited by 19
Bolpagni, M., & Gabrielli, S. (2025) · Informatics · doi:10.3390/informatics12010033 · Cited by 19
Assessing Usability of Large Language Models in EducationPreprint
Huovinen, L. (2024) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/p54gk · Cited by 19
Xie, Z. (2025) · Social Media + Society · doi:10.1177/20563051251323391 · Cited by 18
Human-Robot Interaction Conversational User Enjoyment Scale (HRI CUES)
Irfan, B., Miniota, J., Thunberg, S., Lagerstedt, E., Kuoppamäki, S., Skantze, G., & Pereira, A. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3590359 · Cited by 17
Hsu, L.-J., Khoo, W. Y. H., Swaminathan, M., Amon, K. J., Muralidharan, R., Satov, H., Thant, M. M., Kim, A. S., Tsui, K. M., Crandall, D., et al. (2024) · IEEE RO-MAN · doi:10.1109/ro-man60168.2024.10731264 · Cited by 17
Adhantoro, M. S., Gunawan, D., Prayitno, H. J., Riyanti, R. F., Purnomo, E., & Jufriansah, A. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1446590 · Cited by 17
Investigating usability pitfalls in Brazilian and Foreign governmental chatbots
Monteiro, M. D. S., Batista, G. O. D. S., & Salgado, L. C. D. C. (2023) · Journal on Interactive Systems · doi:10.5753/jis.2023.3104 · Cited by 17
Schlesener, E. A., Ziolkowski, M., Wong, S., Westmoreland, B., & Babu, S. V. (2026) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3774779 · Cited by 16
An, P., Zhang, C., Gao, H., Zhou, Z., Xiao, Y., & Zhao, J. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103365 · Cited by 16
Cao, B., & Wen, C. (2025) · Telematics and Informatics · doi:10.1016/j.tele.2025.102263 · Cited by 16
Towards an LLM-based Intelligent Assistant for Industry 5.0
Figliè, R., Turchi, T., Baldi, G., & Mazzei, D. (2024) · CINECA IRIS Institutial research information system (University of Pisa) · Cited by 16
Evaluating ChatGPT as an Agent for Providing Genetic EducationPreprint
Walton, N., Graceffo, S., Sutherland, N., Kozel, B. A., Danford, C. J., & McGrath, S. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.25.564074 · Cited by 16
Artificial Intelligence Synergetic Opportunities in Services: Conversational Systems Perspective
Rozenes, S., & Cohen, Y. (2022) · Applied Sciences · doi:10.3390/app12168363 · Cited by 15
Productivity of incident management with conversational bots-a review
Iparraguirre-Villanueva, O., Obregon-Palomino, L., Pujay-Iglesias, W., Sierra-Liñan, F., & Cabanillas-Carbonell, M. (2023) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v12.i4.pp1543-1556 · Cited by 15
Chang, Y.-H., Guo, Y.-T., Fu, L., Chiu, M., Chiu, H., & Lin, H. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3320939 · Cited by 14
Li, M., Wang, D., Purwanto, E., Selig, T., Zhang, Q., & Liang, H. (2025) · SoftwareX · doi:10.1016/j.softx.2025.102072 · Cited by 14
Bots in Software Development: A Systematic Literature Review and Thematic Analysis
Moguel-Sánchez, R., Palacios, C. S. M., Ocharán-Hernández, J. O., Limón, X., & Sánchez-García, Á. J. (2023) · Programming and Computer Software · doi:10.1134/s0361768823080145 · Cited by 13
Salminen, J. O., Jung, S., Medina, J., Aldous, K. K., Azem, J. Y., Akhtar, W., Häyhänen, E., & Jansen, B. J. J. (2024) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/3681716.3681727 · Cited by 13
When AI Joins the Negotiation Table: Evaluating AI as a Moderator
Kobiella, C., Isroilov, U., & Schmidt, A. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736630 · Cited by 13
Thi Thuy An, N. G. O., Chau, H. K. L., Tran, N. P. N., & Bui, C. T. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.106095 · Cited by 13
Argumentation effect of a chatbot for ethical discussions about autonomous AI scenarios
Hauptmann, C., Krenzer, A., Völkel, J., & Puppe, F. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02074-x · Cited by 13
AI Companions Reduce Loneliness
De Freitas, J., Uguralp, A. K., Uguralp, Z. O., & Stefano, P. (2024) · arXiv · doi:10.48550/arxiv.2407.19096 · Cited by 12
Artificial Intelligence vs Human in Advertisement Text Writing
Yılmaz, A. (2023) · İktisadi İdari ve Siyasal Araştırmalar Dergisi · doi:10.25204/iktisad.1345154 · Cited by 12
Ge, Y., Li, M., Huang, X., Hu, Y., Wang, Q., Sun, X., & Guo, W. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714238 · Cited by 11
Reducing Perceived Transactional Distance in Distance Education: The Impact of the Chatbot
Onat, O., & Gülseçen, S. (2023) · International Journal of Technology in Education and Science · doi:10.46328/ijtes.511 · Cited by 11
Umami, I., Pee, A. N. C., Sulaiman, H. A., & Khaerudin, A. (2023) · Register Jurnal Ilmiah Teknologi Sistem Informasi · doi:10.26594/register.v9i1.3061 · Cited by 11
Raut, G., Goel, A., & Taneja, U. (2024) · Personal and Ubiquitous Computing · doi:10.1007/s00779-024-01814-8 · Cited by 11
Strategies to Enhance Literacy and Access to Muhammadiyah Information through ChatMu Innovation
Adhantoro, M. S., Gunawan, D., Prayitno, H. J., Riyanti, R. F., & Jufriansah, A. (2024) · International Journal of Religion · doi:10.61707/7hqaep83 · Cited by 11
Design Implications for Next Generation Chatbots with Education 5.0
Sedrakyan, G., Borsci, S., van den Berg, S. M., van Hillegersberg, J., & Veldkamp, B. P. (2024) · Lecture notes in educational technology · doi:10.1007/978-981-97-3883-0_1 · Cited by 11
Self-citationVoice orientation of conversational interfaces in vehicles
Lee, K. M., Moon, Y., Park, I., & Lee, J.-G. (2023) · Behaviour and Information Technology · doi:10.1080/0144929x.2023.2166870 · Cited by 11
Kim, J., Won, J., & Lee, Y. (2025) · Nurse Education in Practice · doi:10.1016/j.nepr.2025.104536 · Cited by 10
Šumak, B., Pušnik, M., Kožuh, I., Šorgo, A., & Brdnik, S. (2025) · Applied Sciences · doi:10.3390/app15020631 · Cited by 10
Zylowski, T., Patrício, N. S., Hettmann, W., Anderer, K., Fischer, K., Wölfel, M., & Henning, P. A. (2024) · International Conference on Education Technology and Computer · doi:10.1145/3702163.3702409 · Cited by 10
Rossouw, A., & Smuts, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40113-8_17 · Cited by 10
Lu, C.-H., Li, C., & Chen, Y.-X. (2026) · Internet Research · doi:10.1108/intr-03-2024-0421 · Cited by 9
Proposing Usability-UX technologies for the design and evaluation of text-based chatbots
Mafra, M. G. S., Nunes, K., da Rocha, S. V., Bráz, G., Silva, A. C., Viana, D., Silva, W., & Rivero, L. (2024) · Journal on Interactive Systems · doi:10.5753/jis.2024.3856 · Cited by 9
AI POWERED CHATBOTS VS HUMAN AGENTS, A COMPATIBLE STUDY ON CUSTOMER SATISFACTION
Mangipudi, P. (2025) · INTERNATIONAL JOURNAL OF COMPUTER ENGINEERING & TECHNOLOGY · doi:10.34218/ijcet_16_02_002 · Cited by 9
User Preferences in Conversational AI for Healthcare: Insights from an Interview Study
Joshi, R., Lee, Y. Y., & Bengler, K. J. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736631 · Cited by 9
Metadata
- Authors
- Borsci, S., Malizia, A., Schmettow, M., van der Velde, F., Tariverdiyeva, G., Balaji, D., & Chamberlain, A.
- Affiliations
- Imperial College London (GB)
- University of Pisa (IT)
- University of Twente (NL)
- University of Amsterdam (NL)
- University of Nottingham (GB)
- Corresponding author
- Chamberlain, Alan
- Venue
- Personal and Ubiquitous Computing
- Published
- 2021-07-21
- Volume
- 26
- Issue
- 1
- Pages
- 95-119
- ISSN
- 1617-4909 (ISSN-L)
- 1617-4917
- OpenAlex
- W3187911027
- Cited by
- 266
- References
- 68
- Citations by year
- 2026: 87 · 2025: 78 · 2024: 52 · 2023: 39 · 2022: 9 · 2021: 1
- FWCI
- 8.89
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Personal and Ubiquitous Computing (publishedVersion, cc-by)
- Repository@Nottingham (University of Nottingham) (publishedVersion, other-oa)
- University of Twente Research Information (publishedVersion, cc-by)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion, other-oa)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- University of Twente Research Information (submittedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/T51729X/1)
- Topics
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- chatbots, user satisfaction, accessibility, chatbot evaluation, psychometric properties
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.