All too human? Identifying and mitigating ethical risks of Social AI
Open accessAbstract
This paper presents an overview of the risks and benefits of Social AI, understood as conversational AI systems that cater to human social needs like romance, companionship, or entertainment. Section 1 of the paper provides a brief history of conversational AI systems and introduces conceptual distinctions to help distinguish varieties of Social AI and pathways to their deployment. Section 2 of the paper adds further context via a brief discussion of anthropomorphism and its relevance to assessment of human-chatbot relationships. Section 3 of the paper provides a survey of potential and in some cases demonstrated harms associated with user interactions with Social AI systems. Finally, Section 4 discusses how the benefits and harms of Social AI can best be addressed, with a primary focus on how frameworks from AI ethics can inform their development.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI companions are proliferating, and some users will likely believe that these companions are conscious.
“AI companions are proliferating, and some users will likely believe that these companions are conscious [ 7].”
From Identifying indicators of consciousness in AI systems (2026) · cites this paper as [7]
People's consciousness attributions to AI chatbots are likely to increase soon.
“Given the increasingly anthropomorphic design and sophistication of AI chatbots (including social AI companions, e.g., Replika), people's consciousness attributions to these systems are likely to increase soon (Shevlin, 2024).”
From Are Attributions of Consciousness to AI Chatbots Epistemically Innocent? (2026) · cites this paper as Shevlin, 2024
Cites
Still being checked.
Cited by
Sorted by citations
Adewale, M. D., & Muhammad, U. I. (2025) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-025-00549-4 · Cited by 46 · see citing works
Identifying indicators of consciousness in AI systems
Butlin, P., Long, R., Bayne, T., Bengio, Y., Birch, J., Chalmers, D., Constant, A., Deane, G., Elmoznino, E., Fleming, S. M., et al. (2026) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.10.011 · Cited by 56 · see citing works
Teens, Tech, and Talk: Adolescents’ Use of and Emotional Reactions to Snapchat’s My AI Chatbot
Vanhoffelen, G., Vandenbosch, L., & Schreurs, L. (2025) · Behavioral Sciences · doi:10.3390/bs15081037 · Cited by 40 · see citing works
Saracini, C., Cornejo-Plaza, I., & Cippitani, R. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1662206 · Cited by 36 · see citing works
Technological folie à deux: Feedback Loops Between AI Chatbots and Mental Illness
Dohnány, S., Kurth-Nelson, Z., Spens, E., Luettgau, L., Reid, A., Gabriel, I., Summerfield, C., Shanahan, M., & Nour, M. M. (2025) · arXiv · doi:10.48550/arxiv.2507.19218 · Cited by 26 · see citing works
“Nevertheless, these general-purpose chatbots are not marketed as mental health tools, and consequently their proliferation proceeds with minimal regulatory oversight 16,17 .”
Anthropomorphism, False Beliefs, and Conversational AIs : How Chatbots Undermine Users' Autonomy
Marchegiani, B. (2025) · Journal of Applied Philosophy · doi:10.1111/japp.70008 · Cited by 17 · see citing works
AI and human autonomy: a literature review
Catena, E. (2026) · AI and Ethics · doi:10.1007/s43681-025-00958-4 · Cited by 20 · see citing works
Chatting with bots: AI, speech acts, and the edge of assertion
Williams, I., & Bayne, T. (2024) · Inquiry · doi:10.1080/0020174x.2024.2434874 · Cited by 12 · see citing works
The Three Social Dimensions of Chatbot Technology
Figueroa-Torres, M. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00826-9 · Cited by 13 · see citing works
We need accountability in human–AI agent relationships
Lange, B., Keeling, G., Manzini, A., & McCroskery, A. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00041-7 · Cited by 5 · see citing works
AI chatbots and the loneliness crisis
Shelmerdine, S. C., & Nour, M. M. (2025) · BMJ · doi:10.1136/bmj.r2509 · Cited by 6 · see citing works
Can chatbot companions alleviate loneliness in autistic users? evaluating digital companions
Hollis, A. S., McKenna-Plumley, P. E., Roman, R. B., & McKeown, G. J. (2026) · AI & Society · doi:10.1007/s00146-026-02877-2 · Cited by 6 · see citing works
Campos‐Castillo, C., Kang, X., & Laestadius, L. I. (2026) · JMIR AI · doi:10.2196/80250 · Cited by 4 · see citing works
On the attribution of confidence to large language models
Keeling, G., & Street, W. (2025) · Inquiry · doi:10.1080/0020174x.2025.2450598 · Cited by 6 · see citing works
Holdier, A. G., & Weirich, K. (2025) · Oxford University Press eBooks · doi:10.1093/9780198945215.003.0074 · Cited by 4 · see citing works
Livia: An Emotion-Aware AR Companion Powered by Modular AI Agents and Progressive Memory Compression
Xi, R., & Wang, X. (2026) · Smart innovation, systems and technologies · doi:10.1007/978-3-032-10951-4_3 · Cited by 3 · see citing works
Anthropomorphising AI: Two Modes, Two Errors
Howdle, G. (2026) · Philosophy & Technology · doi:10.1007/s13347-026-01133-1 · Cited by 2 · see citing works
Can LLMs Talk “Sex”? Exploring How AI Models Handle Intimate Conversations
Lai, H. (2025) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1326 · Cited by 1 · see citing works
Shhh! Your proxy is speaking: real persona social AI and the appropriation of likeness
Loewith, C. (2025) · AI & Society · doi:10.1007/s00146-025-02735-7 · Cited by 1 · see citing works
Ethics at the Frontier of Human-AI Relationships
Shevlin, H. (2025) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198940272.013.0009 · Cited by 1 · see citing works
Self-citationThree frameworks for AI mentality
Shevlin, H. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1715835 · Cited by 1 · see citing works
Self-citationThe Anthropomimetic Turn in Contemporary AI
Shevlin, H. (2026) · Springer eBooks · doi:10.1007/978-3-032-11814-1_26 · Cited by 1 · see citing works
Self-citationDeference, development, and large language models: Issues at the edge of sentience
Bayne, T. (2025) · Mind & Language · doi:10.1111/mila.12537 · Cited by 1 · see citing works
Emotional Cues and Misplaced Trust in Artificial Agents
Masotti, J. (2025) · doi:10.1093/9780198945215.003.0062 · Cited by 1 · see citing works
Are Attributions of Consciousness to AI Chatbots Epistemically Innocent?
Peters, U. (2026) · Minds and Machines · doi:10.1007/s11023-026-09795-8
Trojano, G. (2025) · doi:10.1093/9780198945215.003.0058
Navigating Ethical Challenges of AI Chatbots for Sex Therapy
Fu, M., & Chan, W. (2025) · Oxford University Press eBooks · doi:10.1093/9780198945215.003.0075 · Cited by 1 · see citing works
Contextual negation by moral opposition: rethinking the ethics of (Rape) simulations
Mulert, M. M. (2025) · Ethics and Information Technology · doi:10.1007/s10676-025-09878-7
Ryland, H. (2025) · doi:10.1093/9780198945215.003.0059
Emotions Toward AI and the Echoes of a Paradox
Wallbank, R. (2025) · doi:10.1093/9780198945215.003.0069
The Emotional Alignment Design Policy
Schwitzgebel, E., & Sebo, J. (2026) · Topoi · doi:10.1007/s11245-025-10363-5
Limits of the Anthropomorphisation of Discourse in Chatbots
Chatzikoumi, E., & Arias, C. G. (2026) · Postdisciplinary studies in discourse · doi:10.1007/978-3-032-07761-5_5
Lange, B. (2026) · Springer eBooks · doi:10.1007/978-3-658-50335-2_6
Hollis, A. S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790586 · Cited by 1 · see citing works
User Adoption and Interaction with Generative AI Chatbots Embedded in Social Media Platforms
Shrestha, S., Ranjbar, M. J., Karakaya, K., & Gilbert, S. B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31407-9_24
Yongho, C. (2026) · Philosophy & Social Criticism · doi:10.1177/01914537261481238
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849076
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849062
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849069
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849075
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849066
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849064
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849059
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849073
Full Email Manipulation Example
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849074
(2026) · ACM eBooks · doi:10.1145/3849056.3849071
Vignette 2—A Spanner in the Works
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849061
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849065
(2026) · doi:10.1145/3849056.3849057
Vignette 3: Organizational Lobotomy
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849063
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849068
Hardening the Human Operating System
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849070
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849060
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849067
Feldman, P. (2026) · ACM eBooks · doi:10.1145/3849056.3849072
- Authors
- Shevlin, H.
- Affiliations
- University of Cambridge (GB)
- Corresponding author
- Shevlin, Henry
- ORCID
- Shevlin, Henry
- Venue
- Law, Ethics & Technology
- Publisher
- ELS Publishing Co. Limited
- Published
- 2024
- ISSN
- 2959-3077 (ISSN-L)
- 2959-3085
- OpenAlex
- W4396753825
- Cited by
- 58
- References
- 0
- Citations by year
- 2026: 38 · 2025: 18 · 2024: 2
- FWCI
- 24.04
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Law Ethics & Technology (publishedVersion)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- Funders
- Leverhulme Trust (RC-2015-067)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
- Keywords
- social AI, conversational AI, chatbots, ethical risks, anthropomorphism, AI harms, AI ethics framework, companionship, romance, entertainment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
