CoChat

Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents

Open access
Sun, Guangzhi, Zhan, Xiao, Such, Jose · 2024 · ACM Conversational User Interfaces 2024

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.

Abstract

The incorporation of Large Language Models (LLMs) such as the GPT series into diverse sectors including healthcare, education, and finance marks a significant evolution in the field of artificial intelligence (AI). The increasing demand for personalised applications motivated the design of conversational agents (CAs) to possess distinct personas. This paper commences by examining the rationale and implications of imbuing CAs with unique personas, smoothly transitioning into a broader discussion of the personalisation and anthropomorphism of CAs based on LLMs in the LLM era.

Overview

Integrity signals

Registry

Crossref · 10.1145/3640794.3665887

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 31 references

Research works printed but not linked: 33

  1. Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning. PMLR, 337–371.
  2. Timothy W Bickmore, Suzanne E Mitchell, Brian W Jack, Michael K Paasche-Orlow, Laura M Pfeifer, and Julie O’Donnell. 2010. Response to a relational agent by hospital patients with depressive symptoms. Interacting with computers 22, 4 (2010), 289–298.
  3. Katharine Cook Briggs. 1987. Myers-Briggs type indicator. G. Palo Alto Calif. :Consulting Psychologists Press.
  4. Sydeney Butler. 2023. How to Create ChatGPT Personas for Every Occasion. Retrieved January 2024 from https://www.howtogeek.com/881659/how-to-create-chatgpt-personas-for-every-occasion/
  5. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201 (2023). · arXiv
  6. Domino’s. 2014. MEET DOM: THE VIRTUAL VOICE ORDERING ASSISTANT FOR DOMINO’S PIZZA. Retrieved April 7, 2024 from https://ir.dominos.com/news-releases/news-release-details/meet-dom-virtual-voice-ordering-assistant-dominos-pizzar
  7. Thilo Hagendorff. 2023. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. arXiv preprint arXiv:2303.13988 (2023). · arXiv
  8. Thilo Hagendorff, Sarah Fabi, and Michal Kosinski. 2022. Machine intuition: Uncovering human-like intuitive decision-making in GPT-3.5. arXiv preprint arXiv:2212.05206 (2022). · arXiv
  9. Youjin Hwang, Donghoon Shin, Sion Baek, Bongwon Suh, and Joonhwan Lee. 2021. Applying the persona of user’s family member and the doctor to the conversational agents for healthcare. arXiv preprint arXiv:2109.01729 (2021). · arXiv
  10. Hang Jiang, Xiajie Zhang, Xubo Cao, Jad Kabbara, and Deb Roy. 2023. Personallm: Investigating the ability of gpt-3.5 to express personality traits and gender differences. arXiv preprint arXiv:2305.02547 (2023). · arXiv
  11. Lucas Kohnke, Benjamin Luke Moorhouse, and Di Zou. 2023. ChatGPT for language teaching and learning. RELC Journal (2023), 00336882231162868.
  12. Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, and Xin Zhou. 2023. Better zero-shot reasoning with role-play prompting. arXiv preprint arXiv:2308.07702 (2023). · arXiv
  13. Kausik Lakkaraju, Sai Krishna Revanth Vuruma, Vishal Pallagani, Bharath Muppasani, and Biplav Srivastava. 2023. Can LLMs be Good Financial Advisors?: An Initial Study in Personal Decision Making for Optimized Outcomes. arXiv preprint arXiv:2307.07422 (2023). · arXiv
  14. Nadine Lessio and Alexis Morris. 2020. Toward Design Archetypes for Conversational Agent Personality. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 3221–3228.
  15. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055 (2015). · arXiv
  16. Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. arXiv preprint arXiv:1603.06155 (2016). · arXiv
  17. Junfeng Liu, Christopher Symons, and Ranga Raju Vatsavai. 2022. Persona-Based Conversational AI: State of the Art and Challenges. In 2022 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 993–1001.
  18. Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. 2023. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models.
  19. Alex McFarland. 2023. What is a ChatGPT Persona?Retrieved January 2024 from https://www.unite.ai/what-is-a-chatgpt-persona/
  20. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2018. Towards empathetic open-domain conversation models: A new benchmark and dataset. arXiv preprint arXiv:1811.00207 (2018). · arXiv
  21. Janko Roettgers. 2019. How Alexa Got Her Personality. Retrieved January 2024 from https://variety.com/2019/digital/news/alexa-personality-amazon-echo-1203236019/
  22. Mustafa Safdari, Greg Serapio-García, Clément Crepy, Stephen Fitz, Peter Romero, Luning Sun, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2023. Personality traits in large language models. arXiv preprint arXiv:2307.00184 (2023). · arXiv
  23. Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Dallas Card, and David Jurgens. 2023. You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments. arXiv preprint arXiv:2311.09718 (2023). · arXiv
  24. Haoyu Song, Wei-Nan Zhang, Yiming Cui, Dong Wang, and Ting Liu. 2019. Exploiting persona information for diverse generation of conversational responses. arXiv preprint arXiv:1905.12188 (2019). · arXiv
  25. Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. arXiv preprint arXiv:1506.06714 (2015). · arXiv
  26. Guangzhi Sun, Potsawee Manakul, Adian Liusie, Kunat Pipatanakul, Chao Zhang, Phil Woodland, and Mark Gales. 2024. CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models. arXiv:2405.13684 (2024). · arXiv
  27. Richard Sutcliffe. 2023. A Survey of Personality Persona and Profile in Conversational Agents and Chatbots. arxiv:2401.00609 [cs.CL] · arXiv
  28. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine 29, 8 (2023), 1930–1940.
  29. Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564 (2023). · arXiv
  30. Xinchen Yan, Jimei Yang, Kihyuk Sohn, and Honglak Lee. 2016. Attribute2image: Conditional image generation from visual attributes. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14. Springer, 776–791.
  31. Diyi Yang. 2019. Computational Social Roles. Ph. D. Dissertation. Carnegie Mellon University Pittsburgh, PA, USA.
  32. Peixiang Zhong, Chen Zhang, Hao Wang, Yong Liu, and Chunyan Miao. 2020. Towards persona-based empathetic conversational models. arXiv preprint arXiv:2004.12316 (2020). · arXiv
  33. Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. 2023. Exploring ai ethics of chatgpt: A diagnostic analysis. arXiv preprint arXiv:2301.12867 (2023). · arXiv

Cited by

See all 71

Metadata

Authors
Sun, G., Zhan, X., & Such, J.
Affiliations
  • University of Cambridge (GB)
  • King's College London (GB)
  • Department of Informatics, King's College London, United Kingdom and Universitat Politecnica de Valencia, Spain
Venue
ACM Conversational User Interfaces 2024
Publisher
ACM
Published
2024-07-08
Pages
1-6
OpenAlex
W4400396852
Cited by
55
References
16
Citations by year
2026: 20 · 2025: 30 · 2024: 5
FWCI
16.26
Type
conference paper
Language
en
Open access
Locations
  • ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • EPSRC under grant SAIS: Secure AI assistantS (EP/T026723/1)
  • EU-NextGenerationEU through the Spanish government's Plan de Recuperación, Transformación y Resiliencia (INCIBE's strategic SPRINT (Seguridad y Privacidad en Sistemas con Inteligencia Artificial) C063/23)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Physical Sciences › Computer Science › Human-Computer Interaction › Persona Design and Applications
Keywords
large language models, chatbots, personas, personalization, anthropomorphism, AI agents, GPT, human-AI interaction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents. https://app.cochat.ai/papers/10.1145/3640794.3665887
The paper
Sun, G., Zhan, X., & Such, J. (2024). Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents. ACM Conversational User Interfaces 2024. https://doi.org/10.1145/3640794.3665887

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.