Privacy and Data Protection in ChatGPT and Other AI Chatbots: Strategies for Securing User Information
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
The evolution of artificial intelligence (AI) and machine learning (ML) has led to the development of sophisticated large language models (LLMs) that are used extensively in applications such as chatbots. This research investigates the critical issues of data protection and privacy enhancement in the context of LLM-based chatbots, with a focus on OpenAI's ChatGPT. It explores the dual challenges of safeguarding sensitive user information while ensuring the efficiency of machine learning models. It assesses existing Privacy-Enhancing Technologies (PETs) and proposes innovative methods, such as differential privacy, federated learning, and data minimization techniques. The study also includes a survey of Chatbot users to measure their concerns related to Data Privacy with the use of these LLM-based applications. This study is meant to serve as a comprehensive guide for developers, policymakers, and researchers, contributing to the discourse on data protection in artificial intelligence.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Severity of personal data leaks: Concerns about personal data leaks are paramount79.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [79]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 2×
“Verifying and securely sourcing data can ensure the integrity and legality of the data, but it may also limit the amount of data available for training, possibly affecting the performance and usability of the model (Goodfellow, I., et al., 2015).”
k-ANONYMITY: A MODEL FOR PROTECTING PRIVACY
SWEENEY, L. (2002) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488502001648 · Cited by 8,571
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · cited 3×
“The impact on performance would likely be minimal unless the system becomes overloaded with automated queries (Biggio, B., & Roli, F.,2018) (Shokri, R., 2017) (Akhawe, D., 2013) (Xu, Minrui, et al., 2023) undergo regular Cybersecurity audits, there is a clear consensus for AI systems to adhere to data protection regul…”
How to generate and exchange secrets
Yao, A. C.-C. (1986) · 27th Annual Symposium on Foundations of Computer Science (sfcs 1986) · doi:10.1109/sfcs.1986.25 · Cited by 3,733
Differential Privacy: A Survey of Results
Dwork, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79228-4_1 · Cited by 3,542 · cited 4×
“If successful, the attacker could use the extracted model for malicious purposes, potentially undermining the security and integrity of the original system (Li, Haoran, et al., 2023) (Tramèr, F., et al., 2023) (Shokri, R., et al., 2017) (Dwork, C. et al., 2006). v) Data Poisoning: This is a type of attack where the at…”
InfluentialArtificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540
Wild patterns: Ten years after the rise of adversarial machine learning
Biggio, B., & Roli, F. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.023 · Cited by 1,409 · cited 1×
“The impact on performance would likely be minimal unless the system becomes overloaded with automated queries (Biggio, B., & Roli, F.,2018) (Shokri, R., 2017) (Akhawe, D., 2013) (Xu, Minrui, et al., 2023) undergo regular Cybersecurity audits, there is a clear consensus for AI systems to adhere to…”
Stealing Machine Learning Models via Prediction APIs
Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016) · arXiv · doi:10.48550/arxiv.1609.02943 · Cited by 796
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1802.08232 · Cited by 629
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I., & Talwar, K. (2016) · arXiv · doi:10.48550/arxiv.1610.05755 · Cited by 492
Investigating the impact of user trust on the adoption and use of ChatGPT: Survey analysis
Choudhury, A., & Shamszare, H. (2023) · Journal of Medical Internet Research · doi:10.2196/47184 · Cited by 470
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P. S., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.04226 · Cited by 463 · cited 1×
“These concerns have been addressed in various research papers discussing the usage of LLM-based chatbots, such as those by Hariri (2023), Sebastian (2023), and Cao et al. (2023).”
BackgroundThe Emperor's New Security Indicators
Schechter, S. E., Dhamija, R., Ozment, A., & Fischer, I. (2007) · 2007 IEEE Symposium on Security and Privacy (SP '07) · doi:10.1109/sp.2007.35 · Cited by 405 · cited 2×
“While companies like OpenAI have multiple controls around data privacy and also (Schechter, S., 2007).”
Hariri, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.02017 · Cited by 235 · cited 1×
“These concerns have been addressed in various research papers discussing the usage of LLM-based chatbots, such as those by Hariri (2023), Sebastian (2023), and Cao et al. (2023).”
BackgroundThe Moral Character of Cryptographic WorkPreprint
Rogaway, P. (2016) · IACR Cryptology ePrint Archive · Cited by 231
Dual Adversarial Network: Toward Real-World Noise Removal and Noise Generation
Yue, Z., Zhao, Q., Zhang, L., & Meng, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_3 · Cited by 229
Multi-step Jailbreaking Privacy Attacks on ChatGPT
Li, H., Guo, D., Fan, W., Xu, M., Huang, J., Meng, F., & Song, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.272 · Cited by 187
The Privacy and Security Behaviors of Smartphone App Developers
Balebako, R., Marsh, A., Lin, J., Hong, J., & Faith Cranor, L. (2014) · Proceedings 2014 Workshop on Usable Security · doi:10.14722/usec.2014.23006 · Cited by 161
How Secure is Code Generated by ChatGPT?
Khoury, R., Avila, A. R., Brunelle, J., & Camara, B. M. (2023) · 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC) · doi:10.1109/smc53992.2023.10394237 · Cited by 141
Data integrity verification of the outsourced big data in the cloud environment: A survey
Zhou, L., Fu, A., Yu, S., Su, M., & Kuang, B. (2018) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2018.08.003 · Cited by 112
Can we trust the evaluation on ChatGPT?
Aiyappa, R., An, J., Kwak, H., & Ahn, Y.-Y. (2023) · Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023) · doi:10.18653/v1/2023.trustnlp-1.5 · Cited by 72
Sentiment Analysis for Personalized Chatbots in E-Commerce Applications
El-Ansari, A., & Beni‐Hssane, A. (2023) · Wireless Personal Communications · doi:10.1007/s11277-023-10199-5 · Cited by 70
ChatGPT: A Study of AI Language Processing and its Implications
Mattas, P. S. (2023) · International Journal of Research Publication and Reviews · doi:10.55248/gengpi.2023.4218 · Cited by 54
Beyond the Safeguards: Exploring the Security Risks of ChatGPT
Derner, E., & Batistič, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.08005 · Cited by 54
Do ChatGPT and Other AI Chatbots Pose a Cybersecurity Risk?: An Exploratory Study
Sebastian, G. (2023) · International Journal of Security and Privacy in Pervasive Computing · doi:10.4018/ijsppc.320225 · Cited by 51
Self-citationPersuading Patients Using Rhetoric to Improve Artificial Intelligence Adoption: Experimental Study
Sebastian, G., George, A., & Jackson Jr, G. (2023) · Journal of Medical Internet Research · doi:10.2196/41430 · Cited by 48 · cited 1×
“These concerns have been addressed in various research papers discussing the usage of LLM-based chatbots, such as those by Hariri (2023), Sebastian (2023), and Cao et al. (2023).”
Self-citationA Study on Metaverse Awareness, Cyber Risks, and Steps for Increased Adoption
Sebastian, G. (2022) · International Journal of Security and Privacy in Pervasive Computing · doi:10.4018/ijsppc.308785 · Cited by 47
Self-citationBenefits of Blockchain Initiatives for Value-Based Care: Proposed Framework
Zhang, R., George, A., Kim, J., Johnson, V., & Ramesh, B. (2019) · Journal of Medical Internet Research · doi:10.2196/13595 · Cited by 41
Do ChatGPT and Other AI Chatbots Pose a Cybersecurity Risk? - An Exploratory Study
Sebastian, G. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4363843 · Cited by 35 · cited 1×
“These concerns have been addressed in various research papers discussing the usage of LLM-based chatbots, such as those by Hariri (2023), Sebastian (2023), and Cao et al. (2023).”
BackgroundSelf-citationA Descriptive Study on Metaverse: Cybersecurity Risks, Controls, and Regulatory Framework
Sebastian, G. (2022) · International Journal of Security and Privacy in Pervasive Computing · doi:10.4018/ijsppc.315591 · Cited by 34 · cited 1×
“These concerns have been addressed in various research papers discussing the usage of LLM-based chatbots, such as those by Hariri (2023), Sebastian (2023), and Cao et al. (2023).”
Self-citationBadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT
Shi, J., Liu, Y., Zhou, P., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2304.12298 · Cited by 26
Sparks of GPTs in Edge Intelligence for Metaverse: Caching and Inference for Mobile AIGC ServicesPreprint
Xu, M.-R., Niyato, D., Zhang, H.-L., Kang, J.-W., Xiong, Z.-H., Mao, S., & Han, Z. (2023) · arXiv.org · doi:10.48550/arxiv.2304.08782 · Cited by 23
Impact of metaverse in health care: a study from the care giver’s perspective
Sebastian, S. R., & Babu, B. P. (2022) · International Journal of Community Medicine and Public Health · doi:10.18203/2394-6040.ijcmph20223221 · Cited by 18
Dual Adversarial Network: Toward Real-world Noise Removal and Noise GenerationPreprint
Yue, Z., Zhao, Q., Zhang, L., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05946 · Cited by 14
Unleashing GPT on the Metaverse: Savior or Destroyer?Preprint
Zhou, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.13856 · Cited by 14
ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model TriggerPreprint
Li, J., Yang, Y., Wu, Z., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.14475 · Cited by 9
Srivastava, M. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/6vmu5 · Cited by 8
Srivastava, M. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/6vmu5_v1 · Cited by 1
Research works printed but not linked: 13
- Openai, 2021, ChatGPT: Your AI Friend
- Openai, 2021, OpenAI Data Usage Policy
- Openai, 2021, Fine-Tuning Large Language Models: Dataset and Safety. OpenAI Blog
- Ou Zheng, 2023, ChatGPT is on the horizon: Could a large language model be all we need for Intelligent Transportation?
- Danya Sherbini, 2023, Has technological innovation lost the plot? An interview with AI ethicist Dr. Shannon Vallor, Chicago Policy Review (Online)
- Walid Hariri, 2023, Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing
- Karen Renaud, 2023, From ChatGPT to HackGPT: Meeting the Cybersecurity Threat of Generative AI, 64, 1
- G Cormode, 2019, Data Anonymization Revisited, SIGMOD Rec, 41, 6
- C Dwork, 2006, Calibrating Noise to Sensitivity in Private Data Analysis, Theory of Cryptography Conference
- N Papernot, 2016, Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data, ICLR
- I Goodfellow, 2015, Explaining and Harnessing Adversarial Examples, ICLR
- R Shokri, 2017, Membership inference attacks against machine learning models, 2017 IEEE Symposium on Security and Privacy, 3, 10.1109/SP.2017.41 · DOI · Paper page
- D Akhawe, 2013, Here's my cert, so trust me
Cited by
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
Casheekar, A., Lahiri, A., Rath, K., Prabhakar, K. S., & Srinivasan, K. (2024) · Computer Science Review · doi:10.1016/j.cosrev.2024.100632 · Cited by 211
Privacy-Preserving Techniques in Generative AI and Large Language Models: A Narrative Review
Feretzakis, G., Papaspyridis, K., Gkoulalas-Divanis, A., & Verykios, V. S. (2024) · Information · doi:10.3390/info15110697 · Cited by 145
“To strengthen data anonymization, advanced techniques such as k-anonymity, l-diversity, and t-closeness are employed to prevent re-identification attacks.”
InfluentialChatGPT Promises and Challenges in Education: Computational and Ethical Perspectives
Adel, A., Ahsan, A., & Davison, C. (2024) · Education Sciences · doi:10.3390/educsci14080814 · Cited by 108
“This may involve incorporating elements of self-determination theory, such as autonomy, competence, and relatedness, into the design and implementation of AI-driven educational experiences.”
Background
Metadata
- Authors
- Sebastian, G.
- Affiliations
- Georgia Institute of Technology (US)
- Corresponding author
- Sebastian, Glorin
- ORCID
- Sebastian, Glorin
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2023
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W6959719920
- Cited by
- 56
- References
- 25
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 32 · 2023: 12
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- Topics
- Social Sciences › Law › Digitalization, Law, and Regulation
- Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- privacy, data protection, ChatGPT, AI chatbots, personal data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.