Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
The incorporation of Large Language Models (LLMs) such as the GPT series into diverse sectors including healthcare, education, and finance marks a significant evolution in the field of artificial intelligence (AI). The increasing demand for personalised applications motivated the design of conversational agents (CAs) to possess distinct personas. This paper commences by examining the rationale and implications of imbuing CAs with unique personas, smoothly transitioning into a broader discussion of the personalisation and anthropomorphism of CAs based on LLMs in the LLM era.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Vinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400 · cited 3×
“Moreover, the inherent unpredictability of the LLM adds a layer of complexity.”
BackgroundPersonalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1205 · Cited by 1,177 · cited 1×
“While personality traits, such as being "friendly" or "smart," or frameworks like the Myers-Briggs Type Indicator (MBTI) [4], might define certain characteristics shared by groups of individuals, a persona in CAs represents a more complex and consistent identity [36, 55].”
BackgroundCascella, M., Montomoli, J., Bellini, V., & Bignami, E. (2023) · Journal of Medical Systems · doi:10.1007/s10916-023-01925-4 · Cited by 1,120
Computing Machinery and Intelligence (1950)
Turing, A. M. (2021) · Ideas That Created the Future · doi:10.7551/mitpress/12274.003.0016 · Cited by 1,069
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023) · arXiv · doi:10.48550/arxiv.2302.11382 · Cited by 838
Out of One, Many: Using Language Models to Simulate Human Samples
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023) · Political Analysis · doi:10.1017/pan.2023.2 · Cited by 783
The design and implementation of Xiaoice, an empathetic social chatbot
Zhou, L., Gao, J., Li, D., & Shum, H.-Y. (2020) · Computational Linguistics · doi:10.1162/coli_a_00368 · Cited by 777 · cited 3×
“As one example of a project that is guided by user data, persona XiaoIce was designed based on a large scale analysis of human conversations [57].”
BackgroundMethodInfluentialThe Design and Implementation of XiaoIce, an Empathetic Social Chatbot
Zhou, L., Gao, J., Li, D., & Shum, H.-Y. (2018) · arXiv · doi:10.48550/arxiv.1812.08989 · Cited by 777 · cited 3×
“As one example of a project that is guided by user data, persona XiaoIce was designed based on a large scale analysis of human conversations [57].”
BackgroundMethodInfluential
Research works printed but not linked: 33
- Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning. PMLR, 337–371.
- Timothy W Bickmore, Suzanne E Mitchell, Brian W Jack, Michael K Paasche-Orlow, Laura M Pfeifer, and Julie O’Donnell. 2010. Response to a relational agent by hospital patients with depressive symptoms. Interacting with computers 22, 4 (2010), 289–298.
- Katharine Cook Briggs. 1987. Myers-Briggs type indicator. G. Palo Alto Calif. :Consulting Psychologists Press.
- Sydeney Butler. 2023. How to Create ChatGPT Personas for Every Occasion. Retrieved January 2024 from https://www.howtogeek.com/881659/how-to-create-chatgpt-personas-for-every-occasion/
- Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201 (2023). · arXiv
- Domino’s. 2014. MEET DOM: THE VIRTUAL VOICE ORDERING ASSISTANT FOR DOMINO’S PIZZA. Retrieved April 7, 2024 from https://ir.dominos.com/news-releases/news-release-details/meet-dom-virtual-voice-ordering-assistant-dominos-pizzar
- Thilo Hagendorff. 2023. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. arXiv preprint arXiv:2303.13988 (2023). · arXiv
- Thilo Hagendorff, Sarah Fabi, and Michal Kosinski. 2022. Machine intuition: Uncovering human-like intuitive decision-making in GPT-3.5. arXiv preprint arXiv:2212.05206 (2022). · arXiv
- Youjin Hwang, Donghoon Shin, Sion Baek, Bongwon Suh, and Joonhwan Lee. 2021. Applying the persona of user’s family member and the doctor to the conversational agents for healthcare. arXiv preprint arXiv:2109.01729 (2021). · arXiv
- Hang Jiang, Xiajie Zhang, Xubo Cao, Jad Kabbara, and Deb Roy. 2023. Personallm: Investigating the ability of gpt-3.5 to express personality traits and gender differences. arXiv preprint arXiv:2305.02547 (2023). · arXiv
- Lucas Kohnke, Benjamin Luke Moorhouse, and Di Zou. 2023. ChatGPT for language teaching and learning. RELC Journal (2023), 00336882231162868.
- Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, and Xin Zhou. 2023. Better zero-shot reasoning with role-play prompting. arXiv preprint arXiv:2308.07702 (2023). · arXiv
- Kausik Lakkaraju, Sai Krishna Revanth Vuruma, Vishal Pallagani, Bharath Muppasani, and Biplav Srivastava. 2023. Can LLMs be Good Financial Advisors?: An Initial Study in Personal Decision Making for Optimized Outcomes. arXiv preprint arXiv:2307.07422 (2023). · arXiv
- Nadine Lessio and Alexis Morris. 2020. Toward Design Archetypes for Conversational Agent Personality. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 3221–3228.
- Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055 (2015). · arXiv
- Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. arXiv preprint arXiv:1603.06155 (2016). · arXiv
- Junfeng Liu, Christopher Symons, and Ranga Raju Vatsavai. 2022. Persona-Based Conversational AI: State of the Art and Challenges. In 2022 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 993–1001.
- Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. 2023. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models.
- Alex McFarland. 2023. What is a ChatGPT Persona?Retrieved January 2024 from https://www.unite.ai/what-is-a-chatgpt-persona/
- Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2018. Towards empathetic open-domain conversation models: A new benchmark and dataset. arXiv preprint arXiv:1811.00207 (2018). · arXiv
- Janko Roettgers. 2019. How Alexa Got Her Personality. Retrieved January 2024 from https://variety.com/2019/digital/news/alexa-personality-amazon-echo-1203236019/
- Mustafa Safdari, Greg Serapio-García, Clément Crepy, Stephen Fitz, Peter Romero, Luning Sun, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2023. Personality traits in large language models. arXiv preprint arXiv:2307.00184 (2023). · arXiv
- Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Dallas Card, and David Jurgens. 2023. You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments. arXiv preprint arXiv:2311.09718 (2023). · arXiv
- Haoyu Song, Wei-Nan Zhang, Yiming Cui, Dong Wang, and Ting Liu. 2019. Exploiting persona information for diverse generation of conversational responses. arXiv preprint arXiv:1905.12188 (2019). · arXiv
- Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. arXiv preprint arXiv:1506.06714 (2015). · arXiv
- Guangzhi Sun, Potsawee Manakul, Adian Liusie, Kunat Pipatanakul, Chao Zhang, Phil Woodland, and Mark Gales. 2024. CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models. arXiv:2405.13684 (2024). · arXiv
- Richard Sutcliffe. 2023. A Survey of Personality Persona and Profile in Conversational Agents and Chatbots. arxiv:2401.00609 [cs.CL] · arXiv
- Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine 29, 8 (2023), 1930–1940.
- Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564 (2023). · arXiv
- Xinchen Yan, Jimei Yang, Kihyuk Sohn, and Honglak Lee. 2016. Attribute2image: Conditional image generation from visual attributes. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14. Springer, 776–791.
- Diyi Yang. 2019. Computational Social Roles. Ph. D. Dissertation. Carnegie Mellon University Pittsburgh, PA, USA.
- Peixiang Zhong, Chen Zhang, Hao Wang, Yong Liu, and Chunyan Miao. 2020. Towards persona-based empathetic conversational models. arXiv preprint arXiv:2004.12316 (2020). · arXiv
- Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. 2023. Exploring ai ethics of chatgpt: A diagnostic analysis. arXiv preprint arXiv:2301.12867 (2023). · arXiv
Cited by
PersonaCraft: Leveraging language models for data-driven persona development
Jung, S., Salminen, J. O., Aldous, K. K., & Jansen, B. J. J. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103445 · Cited by 78
Cai, R., Janaka, N. N. P. K., Kim, H., Chen, Y., Zhao, S., Huang, Y., & Hsu, D. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713953 · Cited by 56
“To enhance user experiences, LLM-based systems increasingly incorporate personalization through individual user profiles or group personas [41, 83, 92, 104].”
A foundational architecture for AI agents in healthcare
Liu, F., Niu, Y., Zhang, Q., Wang, K., Dong, Z., Wong, I. N., Cheng, L., Li, T., Duan, L., Li, K., et al. (2025) · Cell Reports Medicine · doi:10.1016/j.xcrm.2025.102374 · Cited by 41
Karim, M. M., Khan, S., Van, D. H., Liu, X. L., Wang, C., & Qu, Q. (2025) · Future Internet · doi:10.3390/fi17080353 · Cited by 50
“The emergence of AI agents, particularly those enhanced by recent breakthroughs in LLMs [8,9], represents a paradigmatic shift toward addressing these annotation challenges through intelligent automation [10].”
Creating and Evaluating Personas Using Generative AI: A Scoping Review of 81 Articles
Amin, D., Salminen, J. O., Ahmed, F., Tervola, S. M. H., Sethi, S., & Jansen, B. J. J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790608 · Cited by 41
Metadata
- Authors
- Sun, G., Zhan, X., & Such, J.
- Affiliations
- University of Cambridge (GB)
- King's College London (GB)
- Department of Informatics, King's College London, United Kingdom and Universitat Politecnica de Valencia, Spain
- Venue
- ACM Conversational User Interfaces 2024
- Publisher
- ACM
- Published
- 2024-07-08
- Pages
- 1-6
- OpenAlex
- W4400396852
- Cited by
- 55
- References
- 16
- Citations by year
- 2026: 20 · 2025: 30 · 2024: 5
- FWCI
- 16.26
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- EPSRC under grant SAIS: Secure AI assistantS (EP/T026723/1)
- EU-NextGenerationEU through the Spanish government's Plan de Recuperación, Transformación y Resiliencia (INCIBE's strategic SPRINT (Seguridad y Privacidad en Sistemas con Inteligencia Artificial) C063/23)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Persona Design and Applications
- Keywords
- large language models, chatbots, personas, personalization, anthropomorphism, AI agents, GPT, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents. https://app.cochat.ai/papers/10.1145/3640794.3665887
- The paper
- Sun, G., Zhan, X., & Such, J. (2024). Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents. ACM Conversational User Interfaces 2024. https://doi.org/10.1145/3640794.3665887
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.