Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior
Open accessThe DOI matches its Crossref record; 64 of the 71 references it cites were found in a registry or an index, and 6 are not checked yet; 1 of the 78 sources its claims cite were checked for support.
Abstract
Chatbots with personality have been shown to affect engagement and user subjective satisfaction. Yet, the design of most chatbots focuses on functionality and accuracy rather than an interpersonal communication style. Existing studies on personality-imbued chatbots have mostly assessed the effect of chatbot personality on user preference and satisfaction. However, the influence of chatbot personality on behavioral qualities, such as users’ trust, engagement, and perceived authenticity of the chatbots, is largely unexplored. To bridge this gap, this study contributes: (1) A detailed design of a personality-imbued chatbot used in academic advising. (2) Empirical findings of an experiment with students who interacted with three different versions of the chatbot. Each version, vetted by psychology experts, represents one of the three dominant traits, agreeableness, conscientiousness, and extraversion. The experiment focused on the effect of chatbot personality on trust, authenticity, engagement, and intention to use the chatbot. Furthermore, we assessed whether gender plays a role in students’ perception of the personality-imbued chatbots. Our findings show a positive impact of chatbot personality on perceived chatbot authenticity and intended engagement, while student gender does not play a significant role in the students’ perception of chatbots.
Verified 2026-11-19 · 6 references not checked yet
Overview
CoChat Summary
TL;DR
This paper designs three text-based academic advising chatbots dominated by agreeableness, conscientiousness, or extraversion and tests their effects on students’ trust, perceived authenticity, usage intention, and intended engagement. Chatbot personality significantly affected perceived authenticity and intended engagement, while student gender showed no significant effect on any measured behavior.
Key findings
- Chatbot personality significantly affected perceived authenticity (H2: p = 0.001) and intended engagement (H4: p = 0.002), but not overall trust (H1: p = 0.114) or usage intention (H3: p = 0.114). Pairwise, trust differed significantly only between the conscientious and agreeable chatbots (p = 0.049); authenticity differed significantly between conscientious vs. extroverted and conscientious vs. agreeable (both p = 0.001); engagement differed significantly between conscientious and extroverted (p = 0.001).
- Students trusted the agreeable chatbot most (M = 4.033, Mdn = 4.1), followed by extroverted (M = 3.788) and conscientious (M = 3.595). They intended to engage most with the extroverted chatbot (M = 3.744), followed by agreeable (M = 3.605) and conscientious (M = 2.853). Extroverted and agreeable chatbots were perceived as almost equally authentic (M = 3.953 and 3.922), while the conscientious chatbot was less authentic (M = 3.233).
- Overall preference was agreeable chatbot 40%, extroverted 32.5%, conscientious 27.5%.
- Student gender had no significant influence on trust, perceived authenticity, usage intention, or intended engagement with any chatbot (all H5–H8 sub-hypotheses n.s.).
Methods
- Within-subject experiment with 43 students (48 recruited, 5 removed for missing/corrupt data) from three universities; 67.4% female, 32.6% male; 86% aged 18–25. Participants interacted with three versions of MyAdvisor, each dominated by conscientiousness, extraversion, or agreeableness, with order counterbalanced across three groups; average experiment duration was 50 min.
- Chatbot design followed the Big-Five Inventory, used the same scripted academic advising content across versions, and manipulated personality through language cues. The design was validated by four professional psychologists; chatbots were implemented in Google Dialogflow.
- Post-interaction questionnaire used a five-point Likert scale for trust, authenticity, usage intention, and engagement. Analysis used Kruskal-Wallis H and Mann-Whitney U tests, Cronbach’s alpha above 0.7, and thematic analysis of qualitative responses.
Limitations
- The paper states: interacting with three chatbot versions may have influenced users’ perceptions; participants came from only three geographic locations; most were non-native English speakers interacting with English-speaking chatbots; brief interactions may explain inconclusive trust and usage-intention results; and the sample was female-dominated (N = 29 female, N = 14 male), so replication with a more gender-balanced population is encouraged.
- Evident from the text: the small sample (43) and short, single-session interaction limit generalizability and causal strength; the study used a within-subject design without a between-subjects comparison, and the personality manipulation was validated by psychologists but not by a participant manipulation check.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures












Verification
Claims the paper makes
1 Valid · 2 Not valid · 57 Not checked · 80 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Chatbot personality has been shown to affect trust [11,12], engagement [6], subjective satisfaction [13,14], and consumer behavior [6].”
- Also stated in: § 2.2. Chatbots in Education · plain
“despite the growing evidence for chatbot personality affecting trust [11,12], engagement [6], and subjective satisfaction [13,14]”
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > 1. Introduction
What the sources say
- Supportedcites [11] Braun, M., Mainz, A., Chadowitz, R., Pfleging, B., and Alt, F. (2019, January 4–9). At your service: Designing voice assistant personalities to improve automotive user interfaces. Proceedings of the CHI Conference on Human Factors in Computing Systems, Glasgow, UK. source
The Trust section (our copy) reports that the four assistant characters differed significantly in perceived trustworthiness — Default rated significantly more trustworthy than Friend, Aunt and Butler (T-test, p < .05), with Friend least trustworthy — and that personalized characters were trusted more when the personality matching was correct and less when it was wrong (Figure 4). The Conclusion states that 'personalization has a positive effect on trust and likability if the voice assistant character matches the user's personality'. The source is a voice-assistant (conversational agent) driving study rather than a text chatbot, but it plainly treats assistant personality as a determinant of trust, which is what the citing sentence attributes to it. (LLM, conf=0.85)
- Not checked yetcites [12] Zhou, 2019, Trusting virtual agents: The effect of personality, ACM Trans. Interact. Intell. Syst. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 2.2. Chatbots in Education · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our findings show a positive impact of chatbot personality on perceived chatbot authenticity and intended engagement, while student gender does not play a significant role in the students' perception of chatbots.”
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our findings show a positive impact of chatbot personality on perceived chatbot authenticity and intended engagement, while student gender does not play a significant role in the students' perception of chatbots.”
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our findings show a positive impact of chatbot personality on perceived chatbot authenticity and intended engagement, while student gender does not play a significant role in the students' perception of chatbots.”
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results indicate a significant difference (p = 0.001) between students' perceived authenticity of the different chatbots.”
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > 5.1.2. Testing Hypothesis 2 (Effect of Chatbot Personality on Perceived Authenticity)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
55 more claims
Create a free account to see all 140 claims and to verify the 77 references still open.
- It may state two separate claims as one. 38
- The quote only partly states this claim. 5
- Probably not a substantial claim. 50
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Furthermore, chatbots can enhance clinical reasoning competency among nursing students by providing various materials and scenarios, allowing them to experience clinical judgments [17].”
From Development and effects of a chatbot education program for self-directed learning in nursing students (2025) · cites this paper as [17]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > Sub-Hypothesis U-Value p-Value Result
“H4B.There is a difference between students' intended engagement with the conscientious and agreeable chatbots.607 0.006 n.s.”
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > Sub-Hypothesis U-Value p-Value Result
“H1B.There is a difference between students' trust in the conscientious and agreeable chatbots.696.5 0.049 supported”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > 5.1. Effect of Chatbot Personality on Trust, Authenticity, Usage Intention, and Engagement
“the students intend to use the agreeable chatbot the most (Mdn = 4, M = 3.721, SD = 0.209)”
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > 5.2. Effect of User Gender on Behavior
“agreeable (Mdn = 3.857, M = 4.5, SD = 1.406) chatbots more than their female counterparts (conscientious: Mdn = 2.931, M = 3, SD = 1.58; agreeable: Mdn = 3.655, M = 4)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > 5.1. Effect of Chatbot Personality on Trust, Authenticity, Usage Intention, and Engagement
“the students trusted the agreeable chatbot the most (Mdn = 4.1, M = 4.033, SD = 0.7733)”
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > 5.2. Effect of User Gender on Behavior
“the female students (Mdn = 3.857, M = 4.1, SD = 0.729) trusted the agreeable chatbot more than their male (Mdn = 3.507, M = 4.15, SD = 1.102) counterparts.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > 5.1.3. Testing Hypothesis 3 (Effect of Chatbot Personality on Usage Intention)
“between the students' intention to use the agreeable and extroverted chatbots (p = 0.0591).”
§ Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior > Sub-Hypothesis U-Value p-Value Result
“H3C.There is a difference between students' intention to use the agreeable and extroverted chatbots.864 0.591 n.s.”
Citations
Cites
Use of Ranks in One-Criterion Variance Analysis
Kruskal, W. H., & Wallis, W. A. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483441 · Cited by 13,912
(1999) · Cited by 7,995
Validation of the five-factor model of personality across instruments and observers.
McCrae, R. R., & Costa, P. T. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.52.1.81 · Cited by 6,404
Braun, V., & Clarke, V. (2022) · Encyclopedia of Quality of Life and Well-Being Research · doi:10.1007/978-3-319-69909-7_3470-2 · Cited by 5,890
Developing and validating trust measures for e-commerce: an integrative typology
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002) · Information Systems Research · doi:10.1287/isre.13.3.334.81 · Cited by 5,550
The Different Roles of Satisfaction, Trust, and Commitment in Customer Relationships
Garbarino, E., & Johnson, M. S. (1999) · Journal of Marketing · doi:10.1177/002224299906300205 · Cited by 4,899
The structure of phenotypic personality traits.
Goldberg, L. R. (1993) · American Psychologist · doi:10.1037/0003-066x.48.1.26 · Cited by 4,816
Norman, W. T. (1963) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0040291 · Cited by 2,485
Managing User Trust in B2C e-Services
Gefen, & Straub (2003) · e-Service Journal · doi:10.2979/esj.2003.2.2.7 · Cited by 1,074
Using linguistic cues for the automatic recognition of personality in conversation and text
Mairesse, F., Walker, M. A., Mehl, M. R., & Moore, R. K. (2007) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2349 · Cited by 1,046
Research works printed but not linked: 6
- Nass, I., and Brave, S. (2005). Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship, MIT Press.
- Reeves, B., and Nass, C.I. (1996). The Media Equation: How People Treat Computers, Television, and New Media Like Real People, Cambridge University Press.
- McCrae, R.R., and Costa, P.T. (2008). The five factor theory of personality. Handbook of Personality: Theory and Research, The Guilford Press.
- Heffernan, N.T., and Croteau, E.A. (September, January 30). Web-based evaluations showing differential learning for tutorial strategies employed by the Ms. Lindquist tutor. Proceedings of the International Conference on Intelligent Tutoring Systems, Maceió, Brazil.
- Qin, C., Huang, W., and Hew, K.F. (2020, January 23–27). Using the Community of Inquiry framework to develop an educational chatbot: Lesson learned from a mobile instant messaging learning environment. Proceedings of the 28th International Conference on Computers in Education, Online.
- Hair, F., Hult, G.T.M., Ringle, C.M., and Sarstedt, M. (2016). A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM), SAGE Publications.
Cited by
Sun, L., & Zhou, L. (2024) · Journal of Educational Computing Research · doi:10.1177/07356331241277937 · Cited by 129
“Several studies have pointed out that Gen-AI can significantly improve the academic achievement of college students (Koç, 2024; Kuhail et al., 2022; Qureshi, 2023) by developing their writing skills (Banihashem et al., 2024), critical thinking (Essien et al., 2024), problem-solving skills (Liu et…”
Chatbots and student motivation: a scoping review
Huang, W., Jiang, J., King, R. B., & Fryer, L. K. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00524-2 · Cited by 102
“For example, the chatbots could provide humanised communication with affective interaction (social response theory in Hong et al., 2023; RiPE in Kumar, 2021) and personalities (Big-Five Inventory model in Kuhail et al., 2022).”
Alimamy, S., & Kuhail, M. A. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107711 · Cited by 83
Self-citationYang, Q., & Lee, Y. (2024) · Journal of risk and financial management · doi:10.3390/jrfm17100470 · Cited by 57
Fadden, I. M., García-Alonso, E.-M., & Menéses, E. L. (2024) · Multimodal Technologies and Interaction · doi:10.3390/mti8120106 · Cited by 59
Metadata
- Authors
- Kuhail, M. A., Thomas, J., Alramlawi, S., Shah, S. J. H., & Thornquist, E.
- Affiliations
- Zayed University (AE)
- Liverpool John Moores University (GB)
- Princess Nourah bint Abdulrahman University (SA)
- University of Missouri–Kansas City (US)
- Corresponding author
- Kuhail, Mohammad Amin
- Venue
- Informatics
- Publisher
- MDPI AG
- Published
- 2022-10-10
- Volume
- 9
- Issue
- 4
- Pages
- 81
- ISSN
- 2227-9709 (ISSN-L)
- OpenAlex
- W4303981344
- Cited by
- 59
- References
- 71
- Citations by year
- 2026: 15 · 2025: 15 · 2024: 20 · 2023: 9
- FWCI
- 3.62
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Informatics (publishedVersion, cc-by)
- Liverpool John Moores University (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Social Science Open Access Repository (GESIS – Leibniz Institute for the Social Sciences) (submittedVersion)
- Funders
- Zayed University (R20131)
- Topics
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- chatbot personality, academic advising, agreeableness, conscientiousness, extraversion, user trust, perceived authenticity, user engagement, gender differences, intention to use
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior. https://app.cochat.ai/papers/10.3390/informatics9040081
- The paper
- Kuhail, M. A., Thomas, J., Alramlawi, S., Shah, S. J. H., & Thornquist, E. (2022). Interacting with a Chatbot-Based Advising System: Understanding the Effect of Chatbot Personality and User Gender on Behavior. Informatics. https://doi.org/10.3390/informatics9040081
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.