The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index; 9 of the 22 sources its claims cite were checked for support.
Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory
Open accessAbstract
Large Language Models (LLMs) are increasingly integrating memory functionalities to provide personalized and context-aware interactions. However, user understanding, practices and expectations regarding these memory systems are not yet well understood. This paper presents a thematic analysis of semi-structured interviews with 18 users to explore their mental models of LLM’s Retrieval Augmented Generation (RAG)-based memory, current usage practices, perceived benefits and drawbacks, privacy concerns and expectations for future memory systems. Our findings reveal diverse and often incomplete mental models of how memory operates. While users appreciate the potential for enhanced personalization and efficiency, significant concerns exist regarding privacy, control and the accuracy of remembered information. Users express a desire for granular control over memory generation, management, usage and updating, including clear mechanisms for reviewing, editing, deleting and categorizing memories, as well as transparent insight into how memories and inferred information are used. We discuss design implications for creating more user-centric, transparent, and trustworthy LLM memory systems.
Overview
CoChat Summary
TL;DR
This paper reports a qualitative interview study with 18 Chinese users on their perceptions of LLM RAG-based memory, using ChatGPT as a case study. It finds varied and often incomplete mental models, a privacy calculus of benefits versus risks, and a strong user demand for granular control and transparency across memory generation, management, usage, and updating.
Key findings
- Users held four dominant mental models of LLM memory: (1) a transient, dialogue-specific buffer; (2) an extension of core training data; (3) an active information processing mechanism; and (4) an acknowledged lack of understanding.
- Participants described benefits including enhanced personalization and efficiency, improved continuity and reduced redundancy, and the LLM as a better companion/assistant; concerns included confidentiality and data leakage, profiling and unwanted persuasion, aggregation and re-identification, unauthorized secondary use, and lack of epistemological certainty.
- Protective strategies included strategic privacy disclosure, proactive input obfuscation, and refusal to use or workarounds.
- Challenges spanned inaccurate memories, lack of meaningful control and transparency, and opaque/uncontrolled inferences; design implications called for context-aware modular memory workspaces, transparent interfaces showing memory origin/use, and co-curation through review-and-commit workflows.
Methods
- Semi-structured interviews with 18 participants recruited online, representing academic and professional backgrounds including engineering, life sciences, social sciences, IT, and design; the study was IRB-approved and participants were compensated 100 RMB. Interviews were conducted in Chinese via Tencent Meeting, lasted approximately 30–60 minutes, and were recorded and transcribed; the interview centered on ChatGPT’s memory feature as a case study.
- Thematic analysis was used: two researchers familiarized themselves with the data, inductively generated initial codes, developed a unified codebook, independently coded 20% of the data with Cohen’s Kappa of 0.90, then independently coded the rest; codes were collated into themes and refined through discussion.
Limitations
- Stated: the study centers on Chinese users under Chinese regulations, so regulatory nuances may differ in GDPR- or CCPA-related regions; participants are biased toward young students with higher education and literacy than average; the work primarily targets ChatGPT, with other products and open-source agent frameworks left for future work; cultural nuances are also left for future work.
- Evident: the small qualitative sample and self-report nature limit generalizability, and the single-product focus may not capture other memory implementations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 2 Not valid · 8 Not checked · 56 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“This approach, however, introduces a known trade-off: increasing user control can also increase cognitive load and privacy fatigue [8], potentially diminishing the user's sense of agency [42,43].”
§ Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory > 5.2 Trade-offs in Memory Management and Usage
What the sources say
- Disputedcites [42] 42. Bo Zhang and S Shyam Sundar. 2019. Proactive vs. reactive personalization: Can customization of privacy enhance user experience? International journal of human-computer studies 128 (2019), 86-99. sourceExperimental — accuracy not yet measured
Section 4.3 reports that participants given privacy-customization options had a higher sense of control than those without them; section 5.2 says customization can make users feel more agentic. The Introduction notes that repeated calls for user action can be intrusive, but it does not say they diminish users’ sense of agency. Page numbers are unavailable in our copy. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [43] 43. Shuning Zhang, Ying Ma, YongquanOwen' Hu, Ting Dang, Hong Jia, Xin Yi, and Hewu Li. 2025. From Patient Burdens to User Agency: Designing for Real-Time Protection Support in Online Health Consultations. arXiv preprint arXiv:2508.00328 (2025). source
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Users' perception towards memory reflects their willingness of stronger agency on their memory control, echoing the longdiscussed balancing between users' and systems' agency [19,42].”
§ Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory > 5.2 Trade-offs in Memory Management and Usage
What the sources say
- Supportedcites [19] 19. Hannah Limerick, David Coyle, and James W Moore. 2014. The experience of agency in human-computer interactions: a review. Frontiers in human neuro- science 8 (2014), 643. source
The citation is attached to the balance between users’ and systems’ agency, rather than to the citing authors’ observation about memory. The review’s Computer assistance > Task automation section explicitly discusses balancing computer assistance with human sense of agency; it does not address memory control. (LLM, conf=0.91)
- Partly supportedcites [42] 42. Bo Zhang and S Shyam Sundar. 2019. Proactive vs. reactive personalization: Can customization of privacy enhance user experience? International journal of human-computer studies 128 (2019), 86-99. sourceExperimental — accuracy not yet measured
The Conclusion (page unavailable) explicitly describes a trade-off between human and machine agency, supporting the sentence’s agency-balancing point. The Previous Privacy Intrusion section (page unavailable) links past privacy experiences to users’ sense of control, but does not examine users’ control over their own memory or show that their perceptions of memory reflect a desire for such control. (LLM, conf=0.83)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To overcome this challenge of opaque inference, users demand radical transparency and control.”
§ Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory > 4.3.3 Challenge to Manage Opaque and Uncontrolled Inferences.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A third group of participants, often those with technical exposure, described a dynamic mental model in which memory is the result of an active information-handling process.”
§ Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory > Mental Model 3: memory as an active information processing mechanism.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The most frequently cited benefit was the potential for LLMs to deliver personalized and efficient interactions by remembering user preferences and context.”
§ Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory > 4.2.1 Perceived benefits of LLM memory.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
30 more claims
Create a free account to see all 72 claims and every source verdict.
- Probably not a substantial claim. 28
- It may state two separate claims as one. 19
- The quote only partly states this claim. 1
- Borderline: one of the tests for a substantial claim is unclear. 1
Citations
Cites
Lost in the Middle: How Language Models Use Long Contexts
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00638 · Cited by 1,612 · cited 1×
“However, as conversations lengthened, agent performance degraded due to distraction and information loss—often called “lost in the middle” [21, 31].”
Lost in the Middle: How Language Models Use Long Contexts
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2307.03172 · Cited by 1,599
A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development
Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q.-L., & Tang, Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123618 · Cited by 1,452 · cited 1×
“…recursive summarization and refinement techniques to distill salient information and reduce redundancy [12, 32, 37, 38, 49], as well as selective retrieval-based systems that store memories in unstructured or layered repositories and fetch them based on conversational relevance [3, 27, 34, 41, 48].”
Privacy as a Concept and a Social Issue: A Multidimensional Developmental Theory
Laufer, R. S., & Wolfe, M. (1977) · Journal of Social Issues · doi:10.1111/j.1540-4560.1977.tb01880.x · Cited by 993 · cited 2×
“Participants’ usage around memory primarily centered around a balance between personalization and privacy risks, which reflect their privacy calculus [15].”
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671470 · Cited by 827 · cited 1×
“The capacity to maintain conversational history is a foundational feature of LLM-driven agents, enabling coherent and contextually aware human–AI interaction [10, 20].”
Self-citationTaxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“Additionally, the anthropomorphic presentation of LLMs encourages oversharing, as users overestimate the system’s capabilities and attribute human-like understanding to it [11, 33].”
A Contextual Approach to Privacy Online
Nissenbaum, H. (2011) · Daedalus · doi:10.1162/daed_a_00113 · Cited by 649 · cited 1×
“Alternatively, providing users with more explicit choices and consent can enhance their agency and control [23, 25].”
A Contextual Approach to Privacy Online
Helen, N. (2012) · Digital Enlightenment Yearbook 2012 · doi:10.3233/978-1-61499-057-4-219 · Cited by 445
The role of privacy fatigue in online privacy behavior
Choi, H., Park, J., & Jung, Y. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2017.12.001 · Cited by 427 · cited 1×
“This approach, however, introduces a known trade-off: increasing user control can also increase cognitive load and privacy fatigue [8], potentially diminishing the user’s sense of agency [42, 43].”
Culture's Consequences: Comparing Values, Behaviors, Institutions and Organisations Across Nations
Eckhardt, G. (2002) · Australian Journal of Management · doi:10.1177/031289620202700105 · Cited by 336 · cited 1×
“The difference in memory’s privacy preferences may be subject to the power distance [9] and culture norms [18].”
Research works printed but not linked: 3
- Natã M Barbosa, Zhuohao Zhang, and Yang Wang. 2020. Do Privacy and Security Matter to Everyone? Quantifying and Clustering { User-Centric} Considerations About Smart Home Device Adoption. In Sixteenth Symposium on Usable Privacy and Security (SOUPS 2020). 417–435.
- Google PAIR. 2019. People + AI Guidebook. Technical Report. Google Research. https://design.google/aiguidebook/
- Yijia Shao, Tianshi Li, Weiyan Shi, Yanchen Liu, and Diyi Yang. [n. d.]. PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
Cited by
Wohn, C., Carik, B., Ding, X., Lee, S. W., Kim, Y., & Rho, E. H. R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791319 · Cited by 8
“They also support personalization based on chat history from previous sessions, which requires less effort from users but does not give them as much fine-grained control [131].”
Towards Usable, Privacy Respecting Long-Term Memory for LLM-based Conversational Agents
Malki, L. M. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799198 · Cited by 3
“Users often have incomplete mental models of how non-parametric memory mechanisms function, and perceive themselves as having little control over what the system remembers and how memories are referenced [27].”
InfluentialLi, T., Li, T. J.-J., Yao, Y., et al. (2000) · Time to knit · doi:10.1145/3733816.3787509
Khaliq, A., Aziz, L., Alawad, W. M., Almutairi, L., Kenany, A. A., & Almakayeel, N. M. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08604-4
Designing Proactive Memory Privacy Visualization and Management in LLM-based ChatbotsPreprint
Zhang, S., Ye, L., Li, S., Xing, H., Jingyu, T., Shui, B., Nie, K., liu, P., Yi, X., & Li, H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7288197
Self-citation
Metadata
- Authors
- Zhang, S., Ma, R., Ma, Y., Li, S., Xu, Y., Yi, X., & Li, H.
- Affiliations
- Tsinghua University (CN)
- Aalto University (FI)
- The University of Melbourne (AU)
- Venue
- Proceedings of the 2025 Workshop on Human-Centered AI Privacy and Security
- Publisher
- ACM
- Published
- 2025-10-13
- Pages
- 10-19
- OpenAlex
- W4416273051
- Cited by
- 6
- References
- 33
- Citations by year
- 2026: 6
- FWCI
- 20.15
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- arXiv (Cornell University) (submittedVersion)
- Aaltodoc (Aalto University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Natural Science Foundation of China (62132010)
- Topics
- Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- large language models, retrieval-augmented generation, LLM memory, mental models, memory accuracy, personalization, user expectations, trustworthy AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory. https://app.cochat.ai/papers/10.1145/3733816.3760750
- The paper
- Zhang, S., Ma, R., Ma, Y., Li, S., Xu, Y., Yi, X., & Li, H. (2025). Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory. Proceedings of the 2025 Workshop on Human-Centered AI Privacy and Security. https://doi.org/10.1145/3733816.3760750
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.