Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 25

The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.

Privacy Implications of Retrieval-Based Language Models

Huang, Yangsibo, Gupta, Samyak, Zhong, Zexuan, Li, Kai, Chen, Danqi·2023·Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
DOI 10.18653/v1/2023.emnlp-main.921
Open this paper in CoChat — free account

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2023.emnlp-main.921

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Retrieval-based language models (e.g., kNNLMs) comprise encoders and the datastore as their key components. When tailoring a retrieval-based language model for a privacy-sensitive task, both components may utilize private data. However, a malicious user possessing API access to the model can exploit a data extraction attack in order to reconstruct sensitive information. This study explores the severity of such attacks and suggests novel strategies to mitigate them.
Figure 1 Retrieval-based language models (e.g., kNNLMs) comprise encoders and the datastore as their key components. When tailoring a retrieval-based language model for a privacy-sensitive task, both components may utilize private data. However, a malicious user possessing API access to the model can exploit a data extraction attack in order to reconstruct sensitive information. This study explores the severity of such attacks and suggests novel strategies to mitigate them.CC-BY · From the PDF
Figure 2 Effect of the interpolation coefficient ( λ ) and the number of nearest neighbors ( k ) on k NN-LMs' utility (measured by perplexity, the blue curve) and privacy risk (measured by the number of reconstructed PIIs in the targeted attack, the green curve). We use Enc private as encoders and D private as the datastore.
Figure 2 Effect of the interpolation coefficient ( λ ) and the number of nearest neighbors ( k ) on k NN-LMs' utility (measured by perplexity, the blue curve) and privacy risk (measured by the number of reconstructed PIIs in the targeted attack, the green curve). We use Enc private as encoders and D private as the datastore.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 13 references

Research works printed but not linked: 24

  1. Martin Abadi, Andy Chu, Ian Goodfellow, H Bren- dan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential pri- vacy. In Proceedings of the 2016 ACM SIGSAC con- ference on computer and communications security, pages 308-318.
  2. Simran Arora, Patrick Lewis, Angela Fan, Jacob Kahn, and Christopher Ré. 2022. Reasoning over public and private data in retrieval-based systems. arXiv preprint arXiv:2203.11027. · arXiv
  3. Mislav Balunovic, Dimitar Dimitrov, Nikola Jovanović, and Martin Vechev. 2022. Lamp: Extracting text from gradients with language model priors. Ad- vances in Neural Information Processing Systems, 35:7641-7654.
  4. David Biesner, Rajkumar Ramamurthy, Robin Sten- zel, Max Lübbering, Lars Hillebrand, Anna Ladi, Maren Pielka, Rüdiger Loitz, Christian Bauckhage, and Rafet Sifa. Anonymization of german finan- cial documents using neural network-based language models with contextual word representations. In- ternational Journal of Data Science and Analytics, pages 1-11.
  5. Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from tril- lions of tokens. In International Conference on Ma- chine Learning (ICML). PMLR.
  6. Mia Xu Chen, Benjamin N Lee, Gagan Bansal, Yuan Cao, Shuyuan Zhang, Justin Lu, Jackie Tsay, Yinan Wang, Andrew M Dai, Zhifeng Chen, et al. 2019. Gmail smart compose: Real-time assisted writing. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2287-2295.
  7. Rachel Cummings, Damien Desfontaines, David Evans, Roxana Geambasu, Matthew Jagielski, Yangsibo Huang, Peter Kairouz, Gautam Kamath, Sewoong Oh, Olga Ohrimenko, et al. 2023. Challenges to- wards the next frontier in privacy. arXiv preprint arXiv:2304.06929. · arXiv
  8. Jieren Deng, Yijue Wang, Ji Li, Chenghong Wang, Chao Shang, Hang Liu, Sanguthevar Rajasekaran, and Caiwen Ding. 2021. TAG: Gradient attack on transformer-based language models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3600-3610, Punta Cana, Do- minican Republic. Association for Computational Linguistics. · DOI
  9. Cynthia Dwork, Krishnaram Kenthapadi, Frank McSh- erry, Ilya Mironov, and Moni Naor. 2006a. Our data, ourselves: Privacy via distributed noise generation. In EUROCRYPT, pages 486-503.
  10. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006b. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference. Springer.
  11. Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, and Danqi Chen. 2022. Recov- ering private text in federated learning of language models. In Advances in Neural Information Process- ing Systems (NeurIPS).
  12. Junxian He, Graham Neubig, and Taylor Berg- Kirkpatrick. 2021. Efficient nearest neighbor lan- guage models. In Empirical Methods in Natural Language Processing (EMNLP).
  13. Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. 2022. Are large pre-trained language models leaking your personal information? In Findings of Empirical Methods in Natural Language Processing (EMNLP).
  14. Yangsibo Huang, Zhao Song, Danqi Chen, Kai Li, and Sanjeev Arora. 2020. Texthide: Tackling data pri- vacy in language understanding tasks. In Findings of Empirical Methods in Natural Language Processing (EMNLP).
  15. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lu- cas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Few-shot learning with re- trieval augmented language models. arXiv preprint arXiv:2208.03299. · arXiv
  16. Bryan Klimt and Yiming Yang. 2004. The enron corpus: A new dataset for email classification research. In Eu- ropean Conference on Machine Learning. Springer.
  17. Jakub Konečnỳ, H Brendan McMahan, Felix X Yu, Pe- ter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for im- proving communication efficiency. In NIPS Work- shop on Private Multi-Party Machine Learning.
  18. Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Gold- berg, and Byron C Wallace. 2021. Does bert pre- trained on clinical notes reveal sensitive data? In North American Chapter of the Association for Com- putational Linguistics (NAACL).
  19. Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. 2022. Large language models can be strong differentially private learners. In International Conference on Learning Representations (ICLR).
  20. Chin-Yew Lin. 2004. ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
  21. Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2019. Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE symposium on security and privacy (SP). IEEE.
  22. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture mod- els. In International Conference on Learning Repre- sentations (ICLR).
  23. Sewon Min, Weijia Shi, Mike Lewis, Xilun Chen, Wen- tau Yih, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2023. Nonparametric masked language modeling. In Findings of Association for Computational Linguis- tics (ACL).
  24. Yuta Nakamura, Shouhei Hanaoka, Yukihiro No- mura, Naoto Hayashi, Osamu Abe, Shuntaro Yada, Shoko Wakamiya, and Eiji Aramaki. 2020. Kart: Privacy leakage framework of language models pre-trained with clinical records. arXiv preprint arXiv:2101.00036. · arXiv

Cited by

Metadata

Authors
Huang, Y., Gupta, S., Zhong, Z., Li, K., & Chen, D.
Venue
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Publisher
Association for Computational Linguistics
Published
2023
Pages
14887-14902
DOI
10.18653/v1/2023.emnlp-main.921
Cited by
25
References
13

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.