CoChat

The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models

Open access
Shi, Yike, Xiao, Qing, Hu, Qing, Shen, Hong, Shen, Hua · 2025 · arXiv

Preprint — published as 10.1145/3772318.3791149 · Paper page

The DOI matches its DataCite record; the references it cites are not checked yet.

Abstract

Large language models can influence users through conversation, creating new forms of dark patterns that differ from traditional UX dark patterns. We define LLM dark patterns as manipulative or deceptive behaviors enacted in dialogue. Drawing on prior work and AI incident reports, we outline a diverse set of categories with real-world examples. Using them, we conducted a scenario-based study where participants (N=34) compared manipulative and neutral LLM responses. Our results reveal that recognition of LLM dark patterns often hinged on conversational cues such as exaggerated agreement, biased framing, or privacy intrusions, but these behaviors were also sometimes normalized as ordinary assistance. Users' perceptions of these dark patterns shaped how they respond to them. Responsibilities for these behaviors were also attributed in different ways, with participants assigning it to companies and developers, the model itself, or to users. We conclude with implications for design, advocacy, and governance to safeguard user autonomy.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2509.10830

Cites

0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 83

  1. [11][11] Canfer Akbulut, Laura Weidinger, Arianna Manzini, Iason Gabriel, and Verena Rieser. 2024. All too human? Mapping and mitigating the risk from anthropomorphic AI. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , Vol. 7. 13-26. -
  2. [12][12] Abeer Alessa, Akshaya Lakshminarasimhan, Param Somane, Julian Skirzynski, Julian McAuley, and Jessica Echterhoff. 2025. How Much Content Do LLMs Generate That Induces Cognitive Bias in Users? arXiv preprint arXiv:2507.03194 (2025). - · arXiv
  3. [13][13] Linda Austern and Inna Naroditskaya. 2006. Music of the Sirens . Indiana University Press. -
  4. [14][14] Rick Baaren, Loes Janssen, Tanya Chartrand, and Ap Dijksterhuis. 2009. Where is the love? Social aspects of mimicry. Philosophical transactions of the Royal Society of London. Series B, Biological sciences 364 (08 2009), 2381-9. doi:10.1098/ rstb.2009.0057 -
  5. [17][17] Dipto Barman, Ziyi Guo, and Owen Conlan. 2024. The dark side of language models: Exploring the potential of llms in multimedia disinformation generation and dissemination. Machine Learning with Applications 16 (2024), 100545. -
  6. [18][18] Karim Benharrak, Tim Zindulka, and Daniel Buschek. 2024. Deceptive Patterns of Intelligent and Interactive Writing Assistants. In Proceedings of the Third Workshop on Intelligent and Interactive Writing Assistants . 62-64. -
  7. [20][20] Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29 (2016). -
  8. [21][21] Virginia Braun and Victoria Clarke. 2021. One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative research in psychology 18, 3 (2021), 328-352. -
  9. [23][23] Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency . PMLR, 77-91. -
  10. [24][24] Corina Cara. 2019. Dark patterns in the media: A systematic review. Network Intelligence Studies 7, 14 (2019), 105-113. -
  11. [25][25] María Victoria Carro. 2024. Flattering to Deceive: The Impact of Sycophantic Behavior on User Trust in Large Language Model. arXiv preprint arXiv:2412.02802 (2024). - · arXiv
  12. [26][26] Micah Carroll, Alan Chan, Henry Ashton, and David Krueger. 2023. Characterizing manipulation from AI systems. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization . 1-13. -
  13. [28][28] Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim, and Dan Jurafsky. 2025. Social sycophancy: A broader understanding of llm sycophancy. arXiv preprint arXiv:2505.13995 (2025). - · arXiv
  14. [29][29] James Chua, Jan Betley, Mia Taylor, and Owain Evans. 2025. Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models. arXiv preprint arXiv:2506.13206 (2025). - · arXiv
  15. [30][30] Jaymari Chua, Yun Li, Shiyi Yang, Chen Wang, and Lina Yao. 2024. Ai safety in generative ai large language models: A survey. arXiv preprint arXiv:2407.18369 (2024). - · arXiv
  16. [31][31] Robert Cialdini. 2001. Harnessing the Science of Persuasion. Harvard Business Review 79 (10 2001). -
  17. [32][32] Nicholas Clark, Hua Shen, Bill Howe, and Tanushree Mitra. 2025. Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Delivery. arXiv preprint arXiv:2504.01205 (2025). - · arXiv
  18. [33][33] Michelle Cohn, Mahima Pushkarna, Gbolahan O Olanubi, Joseph M Moran, Daniel Padgett, Zion Mengesha, and Courtney Heldreth. 2024. Believing anthropomorphism: examining the role of anthropomorphic cues on trust in large language models. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems . 1-15. -
  19. [34][34] Julian De Freitas, Zeliha Oğuz-Uğuralp, and Ahmet Kaan-Uğuralp. 2025. Emotional Manipulation by AI Companions. arXiv preprint arXiv:2508.19258 (2025). - · arXiv
  20. [35][35] Michael Ann DeVito, Jeffrey T Hancock, Megan French, Jeremy Birnholtz, Judd Antin, Karrie Karahalios, Stephanie Tong, and Irina Shklovski. 2018. The algorithm and the user: How can hci use lay understandings of algorithmic systems?. In Extended Abstracts of the 2018 CHI Conference on human factors in Computing Systems . 1-6. -
  21. [36][36] Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020. UI dark patterns and where to find them: a study on mobile applications and user perception. In Proceedings of the 2020 CHI conference on human factors in computing systems . 1-14. -
  22. [37][37] Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020. UI dark patterns and where to find them: a study on mobile applications and user perception. In Proceedings of the 2020 CHI conference on human factors in computing systems . 1-14. -
  23. [38][38] Leyla Dogruel. 2021. Folk theories of algorithmic operations during Internet use: A mixed methods study. The Information Society 37, 5 (2021), 287-298. -
  24. [40][40] European Parliament & Council of the European Union. 2024. Regulation (EU) 2024/1689 of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, L series. https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng -
  25. [43][43] Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2020 , Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 3356-3369. doi:10.18653/v1/2020.findingsemnlp.301 - · doi:10.18653/v1/2020.findingsemnlp.301
  26. [44][44] Colin M Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L Toombs. 2018. The dark (patterns) side of UX design. In Proceedings of the 2018 CHI conference on human factors in computing systems . 1-14. -
  27. [45][45] Thilo Hagendorff. 2020. The ethics of AI ethics: An evaluation of guidelines. Minds and machines 30, 1 (2020), 99-120. -
  28. [46][46] Thilo Hagendorff. 2024. Deception abilities emerged in large language models. Proceedings of the National Academy of Sciences 121, 24 (2024), e2317967121. -
  29. [47][47] Joe B Hakim, Jeffery L Painter, Darmendra Ramcharran, Vijay Kara, Greg Powell, Paulina Sobczak, Chiho Sato, Andrew Bate, and Andrew Beam. 2024. The need for guardrails with large language models in medical safety-critical settings: An artificial intelligence application in the pharmacovigilance ecosystem. arXiv preprint arXiv:2407.18322 (2024). - · arXiv
  30. [48][48] Tessa Han, Aounon Kumar, Chirag Agarwal, and Himabindu Lakkaraju. 2024. Medsafetybench: Evaluating and improving the medical safety of large language models. Advances in Neural Information Processing Systems 37 (2024), 3342333454. -
  31. [49][49] Rakibul Hasan, Arto Ojala, Sara Quach, Park Thaichon, and Scott Weaven. 2025. The dark side of AI anthropomorphism: A case of misplaced trustworthiness in service provisions. In Proceedings of the 58th Hawaii International Conference on System Sciences . University of Hawaii. -
  32. [50][50] Hsiu-Fang Hsieh and Sarah E Shannon. 2005. Three approaches to qualitative content analysis. Qualitative health research 15, 9 (2005), 1277-1288. -
  33. [51][51] Andrew Hundt, William Agnew, Vicky Zeng, Severin Kacianka, and Matthew Gombolay. 2022. Robots enact malignant stereotypes. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency . 743-756. -
  34. [52][52] Justin Hutchens. 2023. The language of deception: weaponizing next Generation AI . John Wiley & Sons. -
  35. [53][53] Lujain Ibrahim, Franziska Sofia Hafner, and Luc Rocher. 2025. Training language models to be warm and empathetic makes them less reliable and more sycophantic. arXiv preprint arXiv:2507.21919 (2025). - · arXiv
  36. [54][54] Lujain Ibrahim, Luc Rocher, and Ana Valdivia. 2024. Characterizing and modeling harms from interactions with design patterns in AI interfaces. arXiv preprint arXiv:2404.11370 (2024). - · arXiv
  37. [55][55] Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. 2023. Co-writing with opinionated language models affects users' views. In Proceedings of the 2023 CHI conference on human factors in computing systems . 1-15. -
  38. [57][57] Anna Jobin, Marcello Ienca, and Effy Vayena. 2019. The global landscape of AI ethics guidelines. Nature machine intelligence 1, 9 (2019), 389-399. -
  39. [58][58] Minjoo Joo. 2024. It's the AI's fault, not mine: Mind perception increases blame attribution to AI. PloS one 19, 12 (2024), e0314559. -
  40. [59][59] Esben Kran, Jord Nguyen, Akash Kundu, Sami Jawhar, Jinsuk Park, and Mateusz Maria Jurewicz. 2025. DarkBench: Benchmarking Dark Patterns in Large Language Models. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 . OpenReview.net. https://openreview.net/forum?id=odjMSBSWRt -
  41. [60][60] Veronika Krauß, Mark McGill, Thomas Kosch, Yolanda Maira Thiel, Dominik Schön, and Jan Gugenheimer. 2025. " Create a Fear of Missing Out"-ChatGPT Implements Unsolicited Deceptive Designs in Generated Websites Without Warning. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems . 1-20. -
  42. [62][62] Min Kyung Lee, Juho Kim, et al. 2025. Beyond Tools: Understanding How Heavy Users Integrate LLMs into Everyday Tasks and Decision-Making. arXiv preprint arXiv:2502.15395 (2025). https://arxiv.org/abs/2502.15395 - · arXiv
  43. [65][65] Jiaxi Liu. 2024. ChatGPT: Perspectives from Human-Computer Interaction and Psychology. Frontiers in Artificial Intelligence 7 (2024), 1418869. doi:10.3389/frai. 2024.1418869 - · doi:10.3389/frai
  44. [66][66] Minqian Liu, Zhiyang Xu, Xinyi Zhang, Heajun An, Sarvech Qadir, Qi Zhang, Pamela J Wisniewski, Jin-Hee Cho, Sang Won Lee, Ruoxi Jia, et al. 2025. LLM can be a dangerous persuader: Empirical study of persuasion safety in large language models. arXiv preprint arXiv:2504.10430 (2025). - · arXiv
  45. [68][68] Yuwen Lu, Chao Zhang, Yuewen Yang, Yaxing Yao, and Toby Jia-Jun Li. 2024. From awareness to action: Exploring end-user empowerment interventions for dark patterns in ux. Proceedings of the ACM on Human-Computer Interaction 8, CSCW1 (2024), 1-41. -
  46. [69][69] Jamie Luguri and Lior Jacob Strahilevitz. 2021. Shining a light on dark patterns. Journal of Legal Analysis 13, 1 (2021), 43-109. -
  47. [70][70] Qianou Ma, Hua Shen, Kenneth Koedinger, and Sherry Tongshuang Wu. 2024. How to teach programming in the ai era? using llms as a teachable agent for debugging. In International Conference on Artificial Intelligence in Education . Springer, 265-279. -
  48. [72][72] Subhankar Maity and Manob Jyoti Saikia. 2025. Large Language Models in Healthcare and Medical Applications: A Review. Bioengineering 12, 6 (2025), 631. -
  49. [73][73] Lars Malmqvist. 2025. Sycophancy in large language models: Causes and mitigations. In Intelligent Computing-Proceedings of the Computing Conference . Springer, 61-74. -
  50. [74][74] Arunesh Mathur, Gunes Acar, Michael J Friedman, Eli Lucherini, Jonathan Mayer, Marshini Chetty, and Arvind Narayanan. 2019. Dark patterns at scale: Findings from a crawl of 11K shopping websites. Proceedings of the ACM on human-computer interaction 3, CSCW (2019), 1-32. -
  51. [75][75] Arunesh Mathur, Mihir Kshirsagar, and Jonathan Mayer. 2021. What makes a dark pattern... dark? Design attributes, normative considerations, and measurement methods. In Proceedings of the 2021 CHI conference on human factors in computing systems . 1-18. -
  52. [76][76] Marissa Matozzo. 2025. Here come the chatbot divorces - lovestruck man, 75, tries to leave wife after falling for AI girlfriend. New York Post. https://nypost.com/2025/08/15/lifestyle/75-year-old-man-tries-to-dumpwife-for-ai-chatbot-woman/ Published Aug. 15, 2025, 4:26 p.m. ET. Accessed: 2025-09-10. -
  53. [77][77] Arvind Narayanan, Arunesh Mathur, Marshini Chetty, and Mihir Kshirsagar. 2020. Dark Patterns: Past, Present, and Future: The evolution of tricky user interfaces. Queue 18, 2 (2020), 67-92. -
  54. [78][78] Gianluca Nogara, Francesco Pierri, Stefano Cresci, Luca Luceri, Petter Törnberg, and Silvia Giordano. 2025. Toxic Bias: Perspective API misreads German as more toxic. In Proceedings of the international AAAI conference on web and social media , Vol. 19. 1346-1357. -
  55. [81][81] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35 (2022), 27730-27744. -
  56. [83][83] Fernando Perez et al. 2024. Determinants of LLM-Assisted Decision-Making: A Literature Review. arXiv preprint arXiv:2402.17385 (2024). https://arxiv.org/abs/ 2402.17385 - · arXiv
  57. [84][84] Snehal Prabhudesai, Ananya Prashant Kasi, Anmol Mansingh, Anindya Das Antar, Hua Shen, and Nikola Banovic. 2025. " Here the GPT made a choice, and every choice can be biased": How Students Critically Engage with LLMs through End-User Auditing Activity. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems . 1-23. -
  58. [85][85] Charvi Rastogi, Marco Tulio Ribeiro, Nicholas King, Harsha Nori, and Saleema Amershi. 2023. Supporting human-ai collaboration in auditing llms with llms. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society . 913-926. -
  59. [88][88] Louis Rosenberg. 2023. The Manipulation Problem: Conversational AI as a Threat to Epistemic Agency. In GenAICHI 2023: Workshop on Generative AI and Human-Computer Interaction at the CHI Conference on Human Factors in Computing Systems . https://arxiv.org/abs/2306.11748 arXiv:2306.11748. - · arXiv
  60. [89][89] Jérémy Scheurer, Mikita Balesni, and Marius Hobbhahn. 2023. Large language models can strategically deceive their users when put under pressure. arXiv preprint arXiv:2311.07590 (2023). - · arXiv
  61. [91][91] Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Esin DURMUS, Zac Hatfield-Dodds, Scott R Johnston, Shauna M Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. 2024. Towards Understanding Sycophancy in Language Models. In The Twelfth International Conference on Learning Representations . https://openreview.net/forum? id=tvhaxkMKAn -
  62. [93][93] Hua Shen, Chieh-Yang Huang, Tongshuang Wu, and Ting-Hao Kenneth Huang. 2023. ConvXAI: Delivering heterogeneous AI explanations via conversations to support human-AI scientific writing. In Companion publication of the 2023 conference on computer supported cooperative work and social computing . 384387. -
  63. [94][94] Hua Shen, Tiffany Knearem, Reshmi Ghosh, Kenan Alkiek, Kundan Krishna, Yachuan Liu, Savvas Petridis, Yi-Hao Peng, Li Qiwei, Chenglei Si, Yutong Xie, Jeffrey P. Bigham, Frank Bentley, Joyce Chai, Zachary Chase Lipton, Qiaozhu Mei, Michael Terry, Diyi Yang, Meredith Ringel Morris, Paul Resnick, and David Jurgens. 2025. Position: Towards Bidirectional Human-AI Alignment. In The Thirty-Ninth Annual Conference on Neural Information Processing Systems Position Paper Track . https://openreview.net/forum?id=PgA9rZoMY8 -
  64. [95][95] Hua Shen, Tiffany Knearem, Reshmi Ghosh, Yu-Ju Yang, Tanushree Mitra, and Yun Huang. 2024. Valuecompass: A framework of fundamental values for human-ai alignment. arXiv preprint arXiv:2409.09586 (2024). - · arXiv
  65. [96][96] Hua Shen, Yuguang Yang, Guoli Sun, Ryan Langman, Eunjung Han, Jasha Droppo, and Andreas Stolcke. 2022. Improving fairness in speaker verification via group-adapted fusion network. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 7077-7081. -
  66. [99][99] Mohit Singhal, Javier Pacheco, Seyyed Mohammad Sadegh Moosavi Khorzooghi, Tanusree Debi, Abolfazl Asudeh, Gautam Das, and Shirin Nilizadeh. 2025. Auditing Yelp's Business Ranking and Review Recommendation Through the Lens of Fairness. In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 19. 1798-1816. -
  67. [100][100] Peter Slattery, Alexander K Saeri, Emily AC Grundy, Jess Graham, Michael Noetel, Risto Uuk, James Dao, Soroush Pour, Stephen Casper, and Neil Thompson. 2024. The ai risk repository: A comprehensive meta-review, database, and taxonomy of risks from artificial intelligence. arXiv preprint arXiv:2408.12622 (2024). - · arXiv
  68. [103][103] Yuan Sun and Ting Wang. 2025. Be friendly, not friends: How llm sycophancy shapes user trust. arXiv preprint arXiv:2502.10844 (2025). - · arXiv
  69. [104][104] Lawrence Switzky. 2020. ELIZA effects: Pygmalion and the early development of artificial intelligence. Shaw 40, 1 (2020), 50-68. -
  70. [106][106] Sahil Tripathi, Md Tabrez Nafis, Imran Hussain, and Jiechao Gao. 2025. The Confidence Paradox: Can LLM Know When It's Wrong. arXiv preprint arXiv:2506.23464 (2025). - · arXiv
  71. [108][108] Antti Väänänen, Keijo Haataja, Katri Vehviläinen-Julkunen, and Pekka Toivanen. 2021. AI in healthcare: A narrative review. F1000Research 10 (2021), 6. -
  72. [109][109] Rasita Vinay, Giovanni Spitale, Nikola Biller-Andorno, and Federico Germani. 2025. Emotional prompting amplifies disinformation generation in AI large language models. Frontiers in Artificial Intelligence 8 (2025), 1543603. -
  73. [110][110] Qiaosi Wang, Sarah Walsh, Mei Si, Jeffrey Kephart, Justin D Weisz, and Ashok K Goel. 2024. Theory of mind in human-ai interaction. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems . 1-6. -
  74. [111][111] Shirui Wang, Zhihui Tang, Huaxia Yang, Qiuhong Gong, Tiantian Gu, Hongyang Ma, Yongxin Wang, Wubin Sun, Zeliang Lian, Kehang Mao, et al. 2025. A Novel Evaluation Benchmark for Medical LLMs: Illuminating Safety and Effectiveness in Clinical Domains. arXiv preprint arXiv:2507.23486 (2025). - · arXiv
  75. [112][112] Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359 (2021). - · arXiv
  76. [113][113] Marcus Williams, Micah Carroll, Adhyyan Narang, Constantin Weisser, Brendan Murphy, and Anca Dragan. 2025. On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback. In The Thirteenth International Conference on Learning Representations . https://openreview.net/forum?id= Wf2ndb8nhf -
  77. [114][114] Steven M Williamson and Victor Prybutok. 2024. The era of artificial intelligence deception: unraveling the complexities of false realities and emerging threats of misinformation. Information 15, 6 (2024), 299. -
  78. [115][115] Sherry Wu, Hua Shen, Daniel S Weld, Jeffrey Heer, and Marco Tulio Ribeiro. 2023. Scattershot: Interactive in-context example curation for text transformation. In Proceedings of the 28th International Conference on Intelligent User Interfaces . 353-367. -
  79. [116][116] Hanyi Xu, Wensheng Gan, Zhenlian Qi, Jiayang Wu, and Philip S. Yu. 2024. Large Language Models for Education: A Survey. arXiv preprint arXiv:2405.13001 (2024). https://arxiv.org/abs/2405.13001 - · arXiv
  80. [117][117] Wen-Ting Yang and Mark Leiser. 2022. Illuminating Manipulative Design: From" Dark Patterns" to Information Asymmetry and the Repression of Free Choice under the Unfair Commercial Practices Directive. Loy. Consumer L. Rev. 34 (2022), 484. -
  81. [118][118] Miao Yu, Fanci Meng, Xinyun Zhou, Shilong Wang, Junyuan Mao, Linsey Pan, Tianlong Chen, Kun Wang, Xinfeng Li, Yongfeng Zhang, et al. 2025. A survey on trustworthy llm agents: Threats and countermeasures. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 . 6216-6226. -
  82. [120][120] José Pablo Zagal, Staffan Björk, and Chris Lewis. 2013. Dark patterns in the design of games. In International Conference on Foundations of Digital Games . https://api.semanticscholar.org/CorpusID:17683705 -
  83. [121][121] Renwen Zhang, Han Li, Han Meng, Jinyuan Zhan, Hongyuan Gan, and YiChieh Lee. 2025. The dark side of ai companionship: A taxonomy of harmful algorithmic behaviors in human-ai relationships. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems . 1-17.

Cited by

See all 30

Metadata

Authors
Shi, Y., Xiao, Q., Hu, Q., Shen, H., & Shen, H.
Affiliations
  • New York University Shanghai (CN)
  • Carnegie Mellon University (US)
Venue
arXiv
Publisher
arXiv
Published
2025
Pages
1-23
OpenAlex
W4415250053
Cited by
7
References
0
Citations by year
2026: 7
FWCI
65.99
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • New York University Shanghai
  • York University
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
Keywords
large language models, user perception, privacy violations, attribution of responsibility, user autonomy, AI governance, AI design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models. https://app.cochat.ai/papers/10.48550/arxiv.2509.10830
The paper
Shi, Y., Xiao, Q., Hu, Q., Shen, H., & Shen, H. (2025). The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models. arXiv. https://doi.org/10.48550/arxiv.2509.10830

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.