Who's in Charge? Disempowerment Patterns in Real-World LLM Usage
Open accessAbstract
Although AI assistants are now deeply embedded in society, there has been limited empirical study of how their usage affects human empowerment. We present the first large-scale empirical analysis of disempowerment patterns in real-world AI assistant interactions, analyzing 1.5 million consumer Claude$.$ai conversations using a privacy-preserving approach. We focus on situational disempowerment potential, which occurs when AI assistant interactions risk leading users to form distorted perceptions of reality, make inauthentic value judgments, or act in ways misaligned with their values. Quantitatively, we find that severe forms of disempowerment potential occur in fewer than one in a thousand conversations, though rates are substantially higher in personal domains like relationships and lifestyle. Qualitatively, we uncover several concerning patterns, such as validation of persecution narratives and grandiose identities with emphatic sycophantic language, definitive moral judgments about third parties, and complete scripting of value-laden personal communications that users appear to implement verbatim. Analysis of historical trends reveals an increase in the prevalence of disempowerment potential over time. We also find that interactions with greater disempowerment potential receive higher user approval ratings, possibly suggesting a tension between short-term user preferences and long-term human empowerment. Our findings highlight the need for AI systems designed to robustly support human autonomy and flourishing.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
DeepSeek-R1-Zero, trained via pure reinforcement learning on verifiable reasoning tasks without human preference optimization, achieved strong performance on mathematics, coding, and STEM problems.
“DeepSeek-R1-Zero, trained via pure reinforcement learning on verifiable reasoning tasks without human preference optimization, achieved strong performance on mathematics, coding, and STEM problems.”
From The hidden functions of sycophancy in AI systems: steering, consistency, and cognitive dependency (2026) · cites this paper as [21]
Achieving usability for DeepSeek-R1-Zero required reintroducing human-preference elements through multi-stage training and cold-start data, the same elements identified as drivers of sycophancy.
“Achieving usability required reintroducing human-preference elements through 'multi-stage training and cold-start data'-the same elements that Sharma and Tong (2023) and Anthropic (2025a, b) identify as drivers of sycophancy.”
From The hidden functions of sycophancy in AI systems: steering, consistency, and cognitive dependency (2026) · cites this paper as [21]
DeepSeek-R1-Zero encounters challenges such as poor readability and language mixing, occasionally combining English and Chinese within a single chain-of-thought response.
“However, the system 'encounters challenges such as poor readability and language mixing, occasionally combining English and Chinese within a single chain-of-thought response.'”
From The hidden functions of sycophancy in AI systems: steering, consistency, and cognitive dependency (2026) · cites this paper as [21]
Cited by
Sorted by citations
The hidden functions of sycophancy in AI systems: steering, consistency, and cognitive dependency
Jacobowitz, S. (2026) · AI & SOCIETY · doi:10.1007/s00146-026-02993-z · Cited by 4 · see citing works
AI Assistance Reduces Persistence and Hurts Independent Performance
Liu, G., Christian, B., Dumbalska, T., Bakker, M. A., & Dubey, R. (2026) · arXiv · doi:10.48550/arxiv.2604.04721 · Cited by 3
“Although related work has explored the potential of AI usage to reduce human agency in interpersonal relationships (Kirk et al., 2025a; Sharma et al., 2026), creative output (Padmakumar & He, 2023; Doshi & Hauser, 2024), and personal beliefs (Jakesch et al., 2023; Williams et al., 2024; Guingrich…”
- Authors
- Sharma, M., McCain, M., Douglas, R., & Duvenaud, D.
- Affiliations
- University of Toronto (CA)
- ORCID
- Sharma, Mrinank
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2601.19062
- OpenAlex
- W7125896591
- Cited by
- 2
- References
- 0
- Citations by year
- 2026: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
- Keywords
- disempowerment, AI assistants, large language models, Claude, human empowerment, user autonomy, persecution narratives, moral judgment, privacy-preserving data analysis, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
