CoChat

Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior

Open access

The DOI matches its Crossref record; 64 of the 71 references it cites were found in a registry or an index, and 6 are not checked yet; 1 of the 78 sources its claims cite were checked for support.

Abstract

Chatbots with personality have been shown to affect engagement and user subjective satisfaction. Yet, the design of most chatbots focuses on functionality and accuracy rather than an interpersonal communication style. Existing studies on personality-imbued chatbots have mostly assessed the effect of chatbot personality on user preference and satisfaction. However, the influence of chatbot personality on behavioral qualities, such as users’ trust, engagement, and perceived authenticity of the chatbots, is largely unexplored. To bridge this gap, this study contributes: (1) A detailed design of a personality-imbued chatbot used in academic advising. (2) Empirical findings of an experiment with students who interacted with three different versions of the chatbot. Each version, vetted by psychology experts, represents one of the three dominant traits, agreeableness, conscientiousness, and extraversion. The experiment focused on the effect of chatbot personality on trust, authenticity, engagement, and intention to use the chatbot. Furthermore, we assessed whether gender plays a role in students’ perception of the personality-imbued chatbots. Our findings show a positive impact of chatbot personality on perceived chatbot authenticity and intended engagement, while student gender does not play a significant role in the students’ perception of chatbots.

Verified 2026-11-19 · 6 references not checked yet

Overview

CoChat Summary

TL;DR

This paper designs three text-based academic advising chatbots dominated by agreeableness, conscientiousness, or extraversion and tests their effects on students’ trust, perceived authenticity, usage intention, and intended engagement. Chatbot personality significantly affected perceived authenticity and intended engagement, while student gender showed no significant effect on any measured behavior.

Key findings

  • Chatbot personality significantly affected perceived authenticity (H2: p = 0.001) and intended engagement (H4: p = 0.002), but not overall trust (H1: p = 0.114) or usage intention (H3: p = 0.114). Pairwise, trust differed significantly only between the conscientious and agreeable chatbots (p = 0.049); authenticity differed significantly between conscientious vs. extroverted and conscientious vs. agreeable (both p = 0.001); engagement differed significantly between conscientious and extroverted (p = 0.001).
  • Students trusted the agreeable chatbot most (M = 4.033, Mdn = 4.1), followed by extroverted (M = 3.788) and conscientious (M = 3.595). They intended to engage most with the extroverted chatbot (M = 3.744), followed by agreeable (M = 3.605) and conscientious (M = 2.853). Extroverted and agreeable chatbots were perceived as almost equally authentic (M = 3.953 and 3.922), while the conscientious chatbot was less authentic (M = 3.233).
  • Overall preference was agreeable chatbot 40%, extroverted 32.5%, conscientious 27.5%.
  • Student gender had no significant influence on trust, perceived authenticity, usage intention, or intended engagement with any chatbot (all H5–H8 sub-hypotheses n.s.).

Methods

  • Within-subject experiment with 43 students (48 recruited, 5 removed for missing/corrupt data) from three universities; 67.4% female, 32.6% male; 86% aged 18–25. Participants interacted with three versions of MyAdvisor, each dominated by conscientiousness, extraversion, or agreeableness, with order counterbalanced across three groups; average experiment duration was 50 min.
  • Chatbot design followed the Big-Five Inventory, used the same scripted academic advising content across versions, and manipulated personality through language cues. The design was validated by four professional psychologists; chatbots were implemented in Google Dialogflow.
  • Post-interaction questionnaire used a five-point Likert scale for trust, authenticity, usage intention, and engagement. Analysis used Kruskal-Wallis H and Mann-Whitney U tests, Cronbach’s alpha above 0.7, and thematic analysis of qualitative responses.

Limitations

  • The paper states: interacting with three chatbot versions may have influenced users’ perceptions; participants came from only three geographic locations; most were non-native English speakers interacting with English-speaking chatbots; brief interactions may explain inconclusive trust and usage-intention results; and the sample was female-dominated (N = 29 female, N = 14 male), so replication with a more gender-balanced population is encouraged.
  • Evident from the text: the small sample (43) and short, single-session interaction limit generalizability and causal strength; the study used a within-subject design without a between-subjects comparison, and the personality manipulation was validated by psychologists but not by a participant manipulation check.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/informatics9040081

Cites

64 of 70 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Figures

Figure 1 The chatbot greets the students and invites them to ask a question. Figure 1. The chatbot greets the students and invites them to ask a question.
Figure 1 The chatbot greets the students and invites them to ask a question. Figure 1. The chatbot greets the students and invites them to ask a question.CC-BY · From the PDF
Figure 2 Manipulating the personalities based on the Big-Five Inventory (BFI) model. Figure 2. Manipulating the personalities based on the Big-Five Inventory (BFI) model.
Figure 2 Manipulating the personalities based on the Big-Five Inventory (BFI) model. Figure 2. Manipulating the personalities based on the Big-Five Inventory (BFI) model.CC-BY · From the PDF
Figure 3 The steps of the study. Figure 3. The steps of the study.
Figure 3 The steps of the study. Figure 3. The steps of the study.CC-BY · From the PDF
Figure 4 Quantitative assessment of trust, authenticity, usage intention, and engagement. Figure 4. Quantitative assessment of trust, authenticity, usage intention, and engagement.
Figure 4 Quantitative assessment of trust, authenticity, usage intention, and engagement. Figure 4. Quantitative assessment of trust, authenticity, usage intention, and engagement.CC-BY · From the PDF
Image 5
CC-BY · From the PDF
Image 6
CC-BY · From the PDF
Figure 5 A box plot for the students ' ratings of the chatbots. Figure 5. Abox plot for the students' ratings of the chatbots.
Figure 5 A box plot for the students ' ratings of the chatbots. Figure 5. Abox plot for the students' ratings of the chatbots.CC-BY · From the PDF
Image 8
CC-BY · From the PDF
Figure 6 A box plot of trust, perceived authenticity, usage intention, and intended engagement ratings for chatbots categorized by gender. Figure 6. A box plot of trust, perceived authenticity, usage intention, and intended engagement ratings for chatbots categorized by gender.
Figure 6 A box plot of trust, perceived authenticity, usage intention, and intended engagement ratings for chatbots categorized by gender. Figure 6. A box plot of trust, perceived authenticity, usage intention, and intended engagement ratings for chatbots categorized by gender.CC-BY · From the PDF
Figure A1. A visualization of the themes of students ' explanation of their ratings of certain chatbot. Figure A1. Avisualization of the themes of students' explanation of their ratings of certain chatbot.
Figure A1. A visualization of the themes of students ' explanation of their ratings of certain chatbot. Figure A1. Avisualization of the themes of students' explanation of their ratings of certain chatbot.CC-BY · From the PDF
Image 11
CC-BY · From the PDF
Image 12
CC-BY · From the PDF

Verification

Claims the paper makes

1 Valid · 2 Not valid · 57 Not checked · 80 Own results and interpretations

55 more claims

Create a free account to see all 140 claims and to verify the 77 references still open.

See all claims — free account
  • It may state two separate claims as one. 38
  • The quote only partly states this claim. 5
  • Probably not a substantial claim. 50

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Show all 70 references

Research works printed but not linked: 6

  1. Nass, I., and Brave, S. (2005). Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship, MIT Press.
  2. Reeves, B., and Nass, C.I. (1996). The Media Equation: How People Treat Computers, Television, and New Media Like Real People, Cambridge University Press.
  3. McCrae, R.R., and Costa, P.T. (2008). The five factor theory of personality. Handbook of Personality: Theory and Research, The Guilford Press.
  4. Heffernan, N.T., and Croteau, E.A. (September, January 30). Web-based evaluations showing differential learning for tutorial strategies employed by the Ms. Lindquist tutor. Proceedings of the International Conference on Intelligent Tutoring Systems, Maceió, Brazil.
  5. Qin, C., Huang, W., and Hew, K.F. (2020, January 23–27). Using the Community of Inquiry framework to develop an educational chatbot: Lesson learned from a mobile instant messaging learning environment. Proceedings of the 28th International Conference on Computers in Education, Online.
  6. Hair, F., Hult, G.T.M., Ringle, C.M., and Sarstedt, M. (2016). A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM), SAGE Publications.

Cited by

See all 74

Metadata

Authors
Kuhail, M. A., Thomas, J., Alramlawi, S., Shah, S. J. H., & Thornquist, E.
Affiliations
  • Zayed University (AE)
  • Liverpool John Moores University (GB)
  • Princess Nourah bint Abdulrahman University (SA)
  • University of Missouri–Kansas City (US)
Corresponding author
Kuhail, Mohammad Amin
Venue
Informatics
Publisher
MDPI AG
Published
2022-10-10
Volume
9
Issue
4
Pages
81
ISSN
2227-9709 (ISSN-L)
OpenAlex
W4303981344
Cited by
59
References
71
Citations by year
2026: 15 · 2025: 15 · 2024: 20 · 2023: 9
FWCI
3.62
Type
journal article
Language
en
Open access
Locations
  • Informatics (publishedVersion, cc-by)
  • Liverpool John Moores University (acceptedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Social Science Open Access Repository (GESIS – Leibniz Institute for the Social Sciences) (submittedVersion)
Funders
Zayed University (R20131)
Topics
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
Keywords
chatbot personality, academic advising, agreeableness, conscientiousness, extraversion, user trust, perceived authenticity, user engagement, gender differences, intention to use

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior. https://app.cochat.ai/papers/10.3390/informatics9040081
The paper
Kuhail, M. A., Thomas, J., Alramlawi, S., Shah, S. J. H., & Thornquist, E. (2022). Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior. Informatics. https://doi.org/10.3390/informatics9040081

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.