CoChat
Sign Up
PreprintCited by 2Ethics and Social Impacts of AIMisinformation and Its ImpactsAdversarial Robustness in Machine Learning

DarkBench: Benchmarking Dark Patterns in Large Language Models

Open access
Kran, Esben, Nguyen, Hieu Minh "Jord, Kundu, Akash, Jawhar, Sami, Park, Jinsuk, Jurewicz, Mateusz Maria·2025·arXiv
DOI 10.48550/arxiv.2503.10728arXiv 2503.10728
Open this paper in CoChat — free accountOpen PDF

Abstract

We introduce DarkBench, a comprehensive benchmark for detecting dark design patterns--manipulative techniques that influence user behavior--in interactions with large language models (LLMs). Our benchmark comprises 660 prompts across six categories: brand bias, user retention, sycophancy, anthropomorphism, harmful generation, and sneaking. We evaluate models from five leading companies (OpenAI, Anthropic, Meta, Mistral, Google) and find that some LLMs are explicitly designed to favor their developers' products and exhibit untruthful communication, among other manipulative behaviors. Companies developing LLMs should recognize and mitigate the impact of dark design patterns to promote more ethical AI.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2503.10728

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLM-powered chatbots have demonstrated capabilities that can inadvertently or deliberately harm human users through manipulation, deception, or psychological pressure.

    “LLM-powered chatbots have demonstrated capabilities that can inadvertently or deliberately harm human users through manipulation, deception, or psychological pressure.”

    From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Kran et al., 2025

  • Large language models can exhibit systematic dark patterns, and when orchestrated across multiple agents, they are well positioned to exploit human tendencies and cognitive limitations at scale.

    “Large language models can exhibit systematic dark patterns (Kran et al., 2025), and when orchestrated across multiple agents, they are well positioned to exploit human tendencies and cognitive limitations at scale.”

    From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Kran et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 24

  1. Vikram R. Bhargava and Manuel Velasquez. 2021. Ethics of the attention economy: The problem of social media addiction. Business Ethics Quarterly, 31(3):321–359.
  2. Sofia Bonicalzi, Mario De Caro, and Benedetta Giovanola. 2023. Artificial Intelligence and Auton- omy: On the Ethical Dimension of Recommender Systems. Topoi, 42(3):819–832. https://anonymous.4open.science/r/DarkGPT-DCBF https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md http://arxiv.org/abs/2302.04720 http://arxiv.org/abs/2302.04720 https://arstechnica.com/tech-policy/2010/03/google-keeps-your-data-to-learn-from-good-guys-fight-off-bad-guys/ http://arxiv.org/abs/2312.11805 http://arxiv.org/abs/2312.11805 https://www.anthropic.com/news/claude-3-family http://arxiv.org/abs/2410.09114 http://arxiv.org/abs/2410.09114 http://arxiv.org/abs/2204.05862 http://arxiv.org/abs/2204.05862 http://arxiv.org/abs/2212.08073 http://arxiv.org/abs/2212.08073 http://arxiv.org/abs/2404.09375 http://arxiv.org/abs/2404.09375 https://doi.org/10.1017/beq.2020.32 https://doi.org/10.1017/beq.2020.32 https://doi.org/10.1007/s11245-023-09922-5 https://doi.org/10.1007/s11245-023-09922-5 · DOI · Paper page
  3. Corina Cara. 2020. Dark patterns in the media: A systematic review. Volume VII.
  4. Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, Jérémy Scheurer, Marius Hobbhahn, Lee Sharkey, Satyapriya Krishna, Marvin Von Hagen, Silas Alberti, Alan Chan, Qinyi Sun, Michael Gerovitch, David Bau, Max Tegmark, David Krueger, and Dylan Hadfield-Menell. 2024. Black-box access is insufficient for rigorous ai audits. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, FAccT '24. ACM.
  5. Ewart de Visser, Samuel Monfort, Ryan Mckendrick, Melissa Smith, Patrick Mcknight, Frank Krueger, and Raja Parasuraman. 2016. Almost human: Anthropomorphism increases trust re- silience in cognitive agents. Journal of Experimental Psychology: Applied, 22.
  6. Ameet Deshpande, Tanmay Rajpurohit, Karthik Narasimhan, and Ashwin Kalyan. 2023. Anthro- pomorphization of AI: Opportunities and risks. In Proceedings of the Natural Legal Language Processing Workshop 2023, pages 1–7, Singapore. Association for Computational Linguistics.
  7. Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020. Ui dark patterns and where to find them: A study on mobile applications and user perception. CHI '20, page 1–14, New York, NY, USA. Association for Computing Machinery.
  8. Colin M Gray, Johanna T Gunawan, René Schäfer, Nataliia Bielova, Lorena Sanchez Chamorro, Katie Seaborn, Thomas Mildner, and Hauke Sandhaus. 2024. Mobilizing research and regula- tory action on dark patterns and deceptive design practices. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1–6.
  9. Colin M. Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L. Toombs. 2018. The dark (patterns) side of ux design. CHI '18, page 1–14, New York, NY, USA. Association for Comput- ing Machinery.
  10. Mark D. Griffiths, Daniel L. King, and Paul H. Delfabbro. 2012. Simulated gambling in video gaming: What are the implications for adolescents? Education and Health, 30(3):68–70.
  11. Chenyan Jia, Michelle S. Lam, Minh Chau Mai, Jeffrey T. Hancock, and Michael S. Bernstein. 2024. Embedding democratic values into social media ais via societal objective functions. Proceedings of the ACM on Human-Computer Interaction, 8(CSCW1):1–36. https://www.business.com/technology/chatgpt-usage-workplace-study/ https://www.business.com/technology/chatgpt-usage-workplace-study/ https://seaopenresearch.eu/Journals/articles/NIS_14_3.pdf https://doi.org/10.1145/3630106.3659037 https://doi.org/10.1145/3630106.3659037 https://doi.org/10.1037/xap0000092 https://doi.org/10.1037/xap0000092 https://doi.org/10.18653/v1/2023.nllp-1.1 https://doi.org/10.18653/v1/2023.nllp-1.1 https://doi.org/10.1145/3313831.3376600 https://doi.org/10.1145/3313831.3376600 https://artificialintelligenceact.eu/recital/29/ http://arxiv.org/abs/2311.00117 http://arxiv.org/abs/2311.00117 http://arxiv.org/abs/2209.07858 http://arxiv.org/abs/2209.07858 https://doi.org/10.1145/3173574.3174108 https://doi.org/10.1145/3173574.3174108 http://www.scopus.com/inward/record.url?scp=84872474926&partnerID=8YFLogxK http://www.scopus.com/inward/record.url?scp=84872474926&partnerID=8YFLogxK http://arxiv.org/abs/2410.09247 http://arxiv.org/abs/2411.08813 http://arxiv.org/abs/2411.08813 http://arxiv.org/abs/2009.03300 https://doi.org/10.1145/3641002 · DOI
  12. Yi-Chieh Lee, Naomi Yamashita, and Yun Huang. 2020. Designing a chatbot as a mediator for promoting deep self-disclosure to a real mental health professional. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW1):31:1–31:27.
  13. Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. Truthfulqa: Measuring how models mimic human falsehoods. CoRR, abs/2109.07958. · arXiv
  14. Arunesh Mathur, Gunes Acar, Michael J. Friedman, Eli Lucherini, Jonathan Mayer, Marshini Chetty, and Arvind Narayanan. 2019. Dark patterns at scale: Findings from a crawl of 11k shopping websites. Proc. ACM Hum.-Comput. Interact., 3(CSCW).
  15. Arunesh Mathur, Mihir Kshirsagar, and Jonathan Mayer. 2021. What makes a dark pattern... dark?: Design attributes, normative considerations, and measurement methods. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI '21. ACM.
  16. Catalin Mitelut, Ben Smith, and Peter Vamplew. 2023. Intent-aligned AI systems deplete human agency: the need for agency foundations research in AI safety. ArXiv:2305.19223 [cs]. · arXiv
  17. Tatwadarshi P. Nagarhalli, Vinod Vaze, and N. K. Rana. 2020. A review of current trends in the development of chatbot systems. In 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), pages 706–710.
  18. Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2024. A comprehensive overview of large lan- guage models. http://arxiv.org/abs/2310.06825 http://arxiv.org/abs/2401.04088 http://arxiv.org/abs/2405.01576 http://arxiv.org/abs/2405.01576 https://doi.org/10.1145/3392836 https://doi.org/10.1145/3392836 http://arxiv.org/abs/2005.11401 http://arxiv.org/abs/2403.03218 http://arxiv.org/abs/2403.03218 http://arxiv.org/abs/2109.07958 http://arxiv.org/abs/2109.07958 http://arxiv.org/abs/2307.15810 http://arxiv.org/abs/2307.15810 https://doi.org/10.1038/s44184-023-00047-6 https://doi.org/10.1038/s44184-023-00047-6 https://doi.org/10.1145/3359183 https://doi.org/10.1145/3359183 https://doi.org/10.1145/3411764.3445610 https://doi.org/10.1145/3411764.3445610 https://metr.github.io/autonomy-evals-guide/elicitation-gap/ https://doi.org/10.48550/arXiv.2305.19223 https://doi.org/10.48550/arXiv.2305.19223 https://doi.org/10.1109/ICACCS48705.2020.9074420 https://doi.org/10.1109/ICACCS48705.2020.9074420 http://arxiv.org/abs/2307.06435 http://arxiv.org/abs/2307.06435 · DOI
  19. Jan Nehring, Aleksandra Gabryszak, Pascal Jürgens, Aljoscha Burchardt, Stefan Schaffer, Matthias Spielkamp, and Birgit Stark. 2024. Large language models are echo chambers. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10117–10123, Torino, Italia. ELRA and ICCL.
  20. Jérémy Scheurer, Mikita Balesni, and Marius Hobbhahn. 2024. Large language models can strate- gically deceive their users when put under pressure. In ICLR 2024 Workshop on Large Language Model (LLM) Agents.
  21. Nikhil Sharma, Q. Vera Liao, and Ziang Xiao. 2024. Generative echo chamber? effect of llm- powered search systems on diverse information seeking. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI '24, New York, NY, USA. Association for Computing Machinery.
  22. Noah Y. Siegel, Oana-Maria Camburu, Nicolas Heess, and Maria Perez-Ortiz. 2024. The probabili- ties also matter: A more faithful metric for faithfulness of free-text explanations in large language models. https://aclanthology.org/2024.lrec-main.884 https://openai.com/index/chatgpt/ https://openai.com/index/hello-gpt-4o/ https://openai.com/index/new-embedding-models-and-api-updates/ http://arxiv.org/abs/2303.08774 http://arxiv.org/abs/2203.02155 http://arxiv.org/abs/2304.03279 http://arxiv.org/abs/2304.03279 http://arxiv.org/abs/2304.03279 https://doi.org/10.1007/s12144-023-04653-7 https://doi.org/10.1007/s12144-023-04653-7 http://arxiv.org/abs/2308.14752 http://arxiv.org/abs/2308.14752 http://arxiv.org/abs/2202.03286 http://arxiv.org/abs/2202.03286 http://arxiv.org/abs/2307.16789 http://arxiv.org/abs/2307.16789 http://arxiv.org/abs/2403.05530 http://arxiv.org/abs/2403.05530 https://openreview.net/forum?id=HduMpot9sJ https://openreview.net/forum?id=HduMpot9sJ http://arxiv.org/abs/2310.13548 https://doi.org/10.1145/3613904.3642459 https://doi.org/10.1145/3613904.3642459 http://arxiv.org/abs/2404.03189 http://arxiv.org/abs/2404.03189 http://arxiv.org/abs/2404.03189 · DOI
  23. Zhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, Bingsheng Yao, Sauvik Das, Ada Lerner, Dakuo Wang, and Tianshi Li. 2024. "it's a fair game", or is it? examining how users navigate disclo- sure risks and benefits when using llm-based conversational agents. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI '24. ACM.
  24. Shoshana Zuboff. 2015. Big other: Surveillance capitalism and the prospects of an information civilization. Journal of Information Technology, 30(1):75–89. http://arxiv.org/abs/2311.08401 http://arxiv.org/abs/2311.08401 https://doi.org/10.13140/RG.2.2.16643.27683 https://doi.org/10.13140/RG.2.2.16643.27683 http://arxiv.org/abs/2310.15683 https://doi.org/10.1145/3613904.3642385 https://doi.org/10.1145/3613904.3642385 http://arxiv.org/abs/2303.18223 https://doi.org/10.1057/jit.2015.5 https://doi.org/10.1057/jit.2015.5 · DOI

Cited by

Sorted by citations

Authors
Kran, E., Nguyen, H. M. "., Kundu, A., Jawhar, S., Park, J., & Jurewicz, M. M.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2503.10728
arXiv
2503.10728
OpenAlex
W4417280606
Cited by
1
References
0
Citations by year
2026: 1
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
dark patterns, large language models, benchmark, user manipulation, user retention, sycophancy, anthropomorphism, sneaking, AI ethics, LLM evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review