PreprintCited by 16Ethics and Social Impacts of AIAI in Service InteractionsSocial Robot Interaction and HRI

The DOI matches its DataCite record.

Be Friendly, Not Friends: How LLM Sycophancy Shapes User Trust

Open access
Sun, Yuan, Wang, Ting·2025·arXiv

Preprint — published as 10.1145/3772318.3791079 · Paper page

Abstract

LLM-powered conversational agents are increasingly influencing our decision-making, raising concerns about "sycophancy" - the tendency for LLMs to excessively agree with users even at the expense of truthfulness. While prior work has primarily examined LLM sycophancy as a model behavior, our understanding of how users perceive this phenomenon and its impact on user trust remains significantly lacking. In this work, we conceptualize LLM sycophancy along two key constructs: conversational demeanor (complimentary vs. neutral) and stance adaptation (adaptive vs. consistent). A 2 x 2 between-subjects experiment (N = 224) revealed complex dynamics: complimentary LLMs that adapted their stance reduced perceived authenticity and trust, while neutral LLMs that adapted enhanced both, suggesting a pathway for manipulating users into over-trusting LLMs beyond their actual capabilities. Our findings advance user-centric understanding of LLM sycophancy and provide profound implications for developing more ethical and trustworthy LLM systems.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2502.10844

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • An emerging body of empirical work finds that when people detect chatbot sycophancy, some respond with heightened skepticism while others accept the chatbot's sycophantic behavior as valid and even desirable.

    “More generally, an emerging body of empirical work (Shi, Xiao, Hu, Shen, & Shen, 2025, §5.2; Sun & Wang, 2025, §4.7; Bo, Kazemitabaar, Deng, Inzlicht, & Anderson, 2025, §4.5; Carro, 2024, §5) finds that when people detect chatbot sycophancy, some respond with heightened skepticism towards the chatbot as expected ('like if a human just always agrees with you, a 'yes man,' you tend not to take them”

    From Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians (2026) · cites this paper as Sun & Wang, 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 80

  1. [1][1] Mohammad Khaled Issa Al Shboul, Asma Alwreikat, and Faiz Abdullah Alotaibi. 2024. Investigating the use of ChatGPT as a novel method for seeking health information: A qualitative approach. Science & Technology Libraries 43, 3 (2024), 225-234. -
  2. [2][2] Lize Alberts, Ulrik Lyngs, and Max Van Kleek. 2024. Computers as bad social actors: Dark patterns and anti-patterns in interfaces that act socially. Proceedings of the ACM on Human-Computer Interaction 8, CSCW1 (2024), 1-25. -
  3. [4][4] Hyuhn-Suhck Bae. 2008. Entertainment-education and recruitment of cornea donors: The role of emotion and issue involvement. Journal of Health Communication 13, 1 (2008), 20-36. -
  4. [5][5] Charles R Berger. 1973. Task performance and attributional communication as determinants of interpersonal attraction. Communications Monographs 40, 4 (1973), 280-286. -
  5. [6][6] Philip Bobko, Leanne Hirshfield, Lucca Eloy, Cara Spencer, Emily Doherty, Jack Driscoll, and Hannah Obolsky. 2023. Human-agent teaming and trust calibration: A theoretical framework, configurable testbed, empirical illustration, and implications for the development of adaptive systems. Theoretical Issues in Ergonomics Science 24, 3 (2023), 310-334. -
  6. [7][7] Konstantinos Bousmalis, Marc Mehu, and Maja Pantic. 2009. Spotting agreement and disagreement: A survey of nonverbal audiovisual cues and tools. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops . IEEE, 1-9. -
  7. [8][8] Jack W Brehm. 1966. A theory of psychological reactance. Academic Press. -
  8. [9][9] Sharon S. Brehm and Jack W. Brehm. 1981. Psychological Reactance: A Theory of Freedom and Control . Academic Press. -
  9. [10][10] Timothy A Brown, Michael T Moore, et al. 2012. Confirmatory factor analysis. Handbook of Structural Equation Modeling 361 (2012), 379. -
  10. [11][11] Judee K Burgoon and Jerold L Hale. 1984. The fundamental topoi of relational communication. Communication Monographs 51, 3 (1984), 193-214. -
  11. [12][12] Shelly Chaiken. 1980. Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology 39, 5 (1980), 752. -
  12. [13][13] Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology 15, 3 (2024), 1-45. -
  13. [14][14] Veena Chattaraman, Wi-Suk Kwon, Juan E Gilbert, and Kassandra Ross. 2019. Should AI-Based, conversational digital assistants employ social-or task-oriented interaction style? A task-competency and reciprocity perspective for older adults. Computers in Human Behavior 90 (2019), 315-330. -
  14. [15][15] Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems 30 (2017). -
  15. [16][16] James Price Dillard and Lijiang Shen. 2005. On the nature of reactance and its role in persuasive health communication. Communication Monographs 72, 2 (2005), 144-168. -
  16. [17][17] Patrick Ehrenbrink. 2020. The role of psychological reactance in human-computer interaction . Springer. -
  17. [18][18] Edgar Erdfelder, Franz Faul, and Axel Buchner. 1996. GPOWER: A general power analysis program. Behavior Research methods, Instruments, & Computers 28 (1996), 1-11. -
  18. [19][19] Susan T Fiske, Amy JC Cuddy, and Peter Glick. 2007. Universal dimensions of social cognition: Warmth and competence. Trends in cognitive sciences 11, 2 (2007), 77-83. -
  19. [21][21] Brian J Fogg and Clifford Nass. 1997. Silicon sycophants: the effects of computers that flatter. International Journal of Human-Computer Studies 46, 5 (1997), 551561. -
  20. [22][22] Marian Friestad and Peter Wright. 1994. The persuasion knowledge model: How people cope with persuasion attempts. Journal of Consumer Research 21, 1 (1994), 1-31. -
  21. [24][24] Aimi Shazwani Ghazali, Jaap Ham, Emilia Barakova, and Panos Markopoulos. 2018. The influence of social cues in persuasive social robots on psychological reactance and compliance. Computers in Human Behavior 87 (2018), 58-65. -
  22. [25][25] Eun Go and S Shyam Sundar. 2019. Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior 97 (2019), 304-316. -
  23. [26][26] William G Graziano and Renée M Tobin. 2002. Agreeableness: Dimension of personality or social desirability artifact? Journal of Personality 70, 5 (2002), 695-728. -
  24. [27][27] Charlotte N Gunawardena. 1995. Social presence theory and implications for interaction and collaborative learning in computer conferences. International Journal of Educational Telecommunications 1, 2 (1995), 147-166. -
  25. [28][28] Fauzia Zahira Munirul Hakim, Lia Maulia Indrayani, and Rosaria Mita Amalia. 2019. A dialogic analysis of compliment strategies employed by Replika chatbot. In Third International Conference of Arts, Language and Culture (ICALC 2018) . Atlantis Press, 266-271. -
  26. [29][29] William Hart, Dolores Albarracín, Alice H Eagly, Inge Brechan, Matthew J Lindberg, and Lisa Merrill. 2009. Feeling validated versus being correct: a metaanalysis of selective exposure to information. Psychological Bulletin 135, 4 (2009), 555. -
  27. [31][31] Kana Higashino, Mitsuhiko Kimoto, Takamasa Iio, Katsunori Shimohara, and Masahiro Shiomi. 2023. Is Politeness Better than Impoliteness? Comparisons of Robot's Encouragement Effects Toward Performance, Moods, and Propagation. International Journal of Social Robotics 15, 5 (2023), 717-729. -
  28. [32][32] Tianran Hu, Anbang Xu, Zhe Liu, Quanzeng You, Yufan Guo, Vibha Sinha, Jiebo Luo, and Rama Akkiraju. 2018. Touch your heart: A tone-aware chatbot for customer care on social media. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems . 1-12. -
  29. [33][33] Ted L Huston. 2013. Foundations of interpersonal attraction . Elsevier. -
  30. [35][35] Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. 2023. Co-writing with opinionated language models affects users' views. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1-15. -
  31. [36][36] Devon Johnson and Kent Grayson. 2005. Cognitive and affective trust in service relationships. Journal of Business Research 58, 4 (2005), 500-507. -
  32. [38][38] Elise Karinshak, Sunny Xun Liu, Joon Sung Park, and Jeffrey T Hancock. 2023. Working with AI to persuade: Examining a large language model's ability to generate pro-vaccination messages. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1-29. -
  33. [40][40] Joshua Klayman. 1995. Varieties of confirmation bias. Psychology of Learning and Motivation 32 (1995), 385-418. -
  34. [42][42] John D Lee and Neville Moray. 1994. Trust, self-confidence, and operators' adaptation to automation. International Journal of Human-Computer Studies 40, 1 (1994), 153-184. -
  35. [43][43] Ruobing Li and S Shyam Sundar. 2022. Can interactive media attenuate psychological reactance to health messages? A study of the role played by user commenting and audience metrics in persuasion. Health Communication 37, 11 (2022), 1355-1367. -
  36. [44][44] Q Vera Liao and S Shyam Sundar. 2022. Designing for responsible trust in AI systems: A communication perspective. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency . 1257-1268. -
  37. [45][45] Leib Litman, Jonathan Robinson, and Tzvi Abberbock. 2017. TurkPrime. com: A versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior Research Methods 49, 2 (2017), 433-442. -
  38. [46][46] Kai Lukoff, Ulrik Lyngs, and Lize Alberts. 2022. Designing to support autonomy and reduce psychological reactance in digital self-Control tools. In SelfDetermination Theory in HCI: Shaping a Research Agenda. Workshop at the ACM CHI Conference on Human Factors in Computing Systems (CHI'22) , Vol. 6. -
  39. [47][47] Yanni Ma, Graham Dixon, and Jay D Hmielowski. 2019. Psychological reactance from reading basic facts on climate change: The role of prior views and political identification. Environmental Communication 13, 1 (2019), 71-86. -
  40. [48][48] Daniel J McAllister. 1995. Affect-and cognition-based trust as foundations for interpersonal cooperation in organizations. Academy of Management Journal 38, 1 (1995), 24-59. -
  41. [49][49] James C McCroskey and Thomas J Young. 1981. Ethos and credibility: The construct and its measurement after three decades. Communication Studies 32, 1 (1981), 24-34. -
  42. [50][50] Claude H Miller, Lindsay T Lane, Leslie M Deatrick, Alice M Young, and Kimberly A Potts. 2007. Psychological reactance and promotional health messages: The effects of controlling language, lexical concreteness, and the restoration of freedom. Human Cmmunication Research 33, 2 (2007), 219-240. -
  43. [51][51] YOUNGME MOON and CLIFFORD NASS. 1996. How 'Real' Are Computer Personalities?: Psychological Responses to Personality Types in Human-Computer Interaction. Communication Research 23, 6 (Dec. 1996), 651-674. doi:10.1177/ 009365096023006002 Publisher: SAGE Publications Inc. -
  44. [52][52] Emily Moyer-Gusé and Robin L Nabi. 2010. Explaining the effects of narrative in an entertainment television program: Overcoming resistance to persuasion. Human Communication Research 36, 1 (2010), 26-52. -
  45. [53][53] Jaya Narain, Tina Quach, Monique Davey, Hae Won Park, Cynthia Breazeal, and Rosalind Picard. 2020. Promoting wellbeing with sunny, a chatbot that facilitates positive messages within social groups. In Extended abstracts of the 2020 CHI Conference on Human Factors in Computing Systems . 1-8. -
  46. [54][54] Clifford Nass, Jonathan Steuer, and Ellen R Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . 72-78. -
  47. [55][55] Mario Neururer, Stephan Schlögl, Luisa Brinkschulte, and Aleksander Groth. 2018. Perceptions on authenticity in chatbots. Multimodal Technologies and Interaction 2, 3 (2018), 60. -
  48. [56][56] Raymond S Nickerson. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology 2, 2 (1998), 175-220. -
  49. [59][59] Daniel J O'Keefe. 1999. How to handle opposing arguments in persuasive messages: A meta-analytic review of the effects of one-sided and two-sided messages. Annals of the International Communication Association 22, 1 (1999), 209-249. -
  50. [60][60] Stefan Palan and Christian Schitter. 2018. Prolific. ac-A subject pool for online experiments. Journal of Behavioral and Experimental Finance 17 (2018), 22-27. -
  51. [61][61] Janak Pandey and Saroj Kakkar. 1982. Supervisors' affect: Attraction and positive evaluation as a function of enhancement of others. Psychological Reports 50, 2 (1982), 479-486. -
  52. [62][62] Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. 2023. Steering llama 2 via contrastive activation addition. arXiv preprint arXiv:2312.06681 (2023). - · arXiv
  53. [63][63] Eyal Peer, Laura Brandimarte, Sonam Samat, and Alessandro Acquisti. 2017. Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology 70 (2017), 153-163. -
  54. [64][64] Ethan Perez, Sam Ringer, Kamil˙ e Lukoši¯ ut˙ e, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. 2022. Discovering language model behaviors with model-written evaluations. arXiv preprint arXiv:2212.09251 (2022). - · arXiv
  55. [65][65] Richard E Petty and John T Cacioppo. 1979. Issue involvement can increase or decrease persuasion by enhancing message-relevant cognitive responses. Journal of Personality and Social Psychology 37, 10 (1979), 1915. -
  56. [66][66] Pearl Pu, Li Chen, and Rong Hu. 2011. A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM Conference on Recommender Systems . 157-164. -
  57. [67][67] Leonardo Ranaldi and Giulia Pucci. 2023. When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviour. arXiv preprint arXiv:2311.09410 (2023). - · arXiv
  58. [68][68] Maaike Roubroeks, Jaap Ham, and Cees Midden. 2011. When artificial social agents try to persuade people: The role of social agency on the occurrence of psychological reactance. International Journal of Social Robotics 3 (2011), 155-165. -
  59. [69][69] Rajat Roy and Vik Naidoo. 2021. Enhancing chatbot effectiveness: The role of anthropomorphic conversational styles and time orientation. Journal of Business Research 126 (2021), 23-34. -
  60. [70][70] Lennart Seitz. 2024. Artificial empathy in healthcare chatbots: Does it feel authentic? Computers in Human Behavior: Artificial Humans 2, 1 (2024), 100067. -
  61. [72][72] Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al. 2023. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548 (2023). - · arXiv
  62. [73][73] Nikhil Sharma, Q Vera Liao, and Ziang Xiao. 2024. Generative echo chamber? Effect of LLM-powered search systems on diverse information seeking. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1-17. -
  63. [74][74] Lijiang Shen. 2015. Antecedents to psychological reactance: The impact of threat, message frame, and choice. Health communication 30, 10 (2015), 975-985. -
  64. [75][75] Heather Shoenberger, Eunjin Kim, and Yuan Sun. 2021. Advertising during COVID-19: Exploring perceived brand message authenticity and potential psychological reactance. Journal of Advertising 50, 3 (2021), 253-261. -
  65. [76][76] John Short, Ederyn Williams, and Bruce Christie. 1976. The social psychology of telecommunications . Wiley, United Kingdom. -
  66. [77][77] Paul J Silvia. 2006. Reactance and the dynamics of disagreement: Multiple paths from threatened freedom to resistance to persuasion. European Journal of Social Psychology 36, 5 (2006), 673-685. -
  67. [79][79] Jack Stilgoe, Richard Owen, and Phil Macnaghten. 2020. Developing a framework for responsible innovation. In The Ethics of Nanotechnology, Geoengineering, and Clean Energy . Routledge, 347-359. -
  68. [80][80] Yuan Sun, Magdalayna Drivas, Mengqi Liao, and S Shyam Sundar. 2023. When recommender systems snoop into social media, users trust them less for health advice. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1-14. -
  69. [81][81] S Shyam Sundar. 2008. The MAIN model: A heuristic approach to understanding technology effects on credibility . MacArthur Foundation Digital Media and Learning Initiative Cambridge, MA. -
  70. [82][82] S Shyam Sundar and Jinyoung Kim. 2019. Machine heuristic: When we trust computers more than humans with our personal information. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems . 1-9. -
  71. [83][83] Gareth Terry, Nikki Hayfield, Victoria Clarke, Virginia Braun, et al. 2017. Thematic analysis. The SAGE handbook of qualitative research in psychology 2, 17-37 (2017), 25. -
  72. [86][86] Amos Tversky and Daniel Kahneman. 1974. Judgment under Uncertainty: heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. Science 185, 4157 (1974), 1124-1131. -
  73. [87][87] Tibert Verhagen, Jaap Van Nes, Frans Feldberg, and Willemijn Van Dolen. 2014. Virtual customer service agents: Using social presence and personalization to shape online service encounters. Journal of Computer-Mediated Communication 19, 3 (2014), 529-545. -
  74. [88][88] Sarah Theres Völkel and Lale Kaya. 2021. Examining user preference for agreeableness in chatbots. In Proceedings of the 3rd Conference on Conversational User Interfaces . 1-6. -
  75. [90][90] Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. 2023. Simple synthetic data reduces sycophancy in large language models. arXiv preprint arXiv:2308.03958 (2023). - · arXiv
  76. [91][91] Magdalena Wischnewski, Nicole Krämer, and Emmanuel Müller. 2023. Measuring and understanding trust calibrations for automated systems: a survey of the stateof-the-art and future directions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1-16. -
  77. [92][92] Ziang Xiao, Wesley Hanwen Deng, Michelle S Lam, Motahhare Eslami, Juho Kim, Mina Lee, and Q Vera Liao. 2024. Human-Centered Evaluation and Auditing of Language Models. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems . 1-6. -
  78. [93][93] Diana Tze Yeong Tan and Ramadhar Singh. 1995. Attitudes and attraction: A developmental study of the similarity-attraction and dissimilarity-repulsion hypotheses. Personality and Social Psychology Bulletin 21, 9 (1995), 975-986. -
  79. [94][94] Xinyan Zhao, Yuan Sun, Wenlin Liu, and Chau-Wai Wong. 2024. Tailoring Generative AI Chatbots for Multiethnic Communities in Disaster Preparedness Communication: Extending the CASA Paradigm. arXiv preprint arXiv:2406.08411 (2024). - · arXiv
  80. [95][95] Jiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G Parker, and Munmun De Choudhury. 2023. Synthetic lies: Understanding AI-generated misinformation and evaluating algorithmic and human solutions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1-20.

Cited by

Sorted by citations

Metadata

Authors
Sun, Y., & Wang, T.
Affiliations
  • University of Florida (US)
  • Stony Brook University (US)
Venue
arXiv
Publisher
arXiv
Published
2025
Pages
1-15
OpenAlex
W4407684130
Cited by
16
References
0
Citations by year
2026: 16
FWCI
150.73
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Science Foundation (2405136, 2406572)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
user trust, chatbots, perceived authenticity, LLM alignment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review