CoChat
Sign Up
PreprintCited by 6Privacy-Preserving Technologies in DataPrivacy, Security, and Data ProtectionEthics and Social Impacts of AI

AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents

Open access
Zharmagambetov, Arman, Guo, Chuan, Evtimov, Ivan, Pavlova, Maya, Salakhutdinov, Ruslan, Chaudhuri, Kamalika·2025·arXiv
DOI 10.48550/arxiv.2503.09780arXiv 2503.09780
Open this paper in CoChat — free accountOpen PDF

Abstract

Autonomous AI agents that can follow instructions and perform complex multi-step tasks have tremendous potential to boost human productivity. However, to perform many of these tasks, the agents need access to personal information from their users, raising the question of whether they are capable of using it appropriately. In this work, we introduce a new benchmark AgentDAM that measures if AI web-navigation agents follow the privacy principle of ``data minimization''. For the purposes of our benchmark, data minimization means that the agent uses a piece of potentially sensitive information only if it is ``necessary'' to complete a particular task. Our benchmark simulates realistic web interaction scenarios end-to-end and is adaptable to all existing web navigation agents. We use AgentDAM to evaluate how well AI agents built on top of GPT-4, Llama-3 and Claude can limit processing of potentially private information, and show that they are prone to inadvertent use of unnecessary sensitive information. We also propose a prompting-based defense that reduces information leakage, and demonstrate that our end-to-end benchmarking provides a more realistic measure than probing LLMs about privacy. Our results highlight that further research is needed to develop AI agents that can prioritize data minimization at inference time.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2503.09780

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 28

  1. [1][1] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security , pages 308-318, 2016. -
  2. [2][2] M. Andriushchenko, A. Souly, M. Dziemian, D. Duenas, M. Lin, J. Wang, D. Hendrycks, A. Zou, Z. Kolter, M. Fredrikson, E. Winsor, J. Wynne, Y. Gal, and X. Davies. AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents. ArXiv , abs/2410.09024, 2024. - · arXiv
  3. [4][4] E. Bagdasarian, R. Yi, S. Ghalebikesabi, P. Kairouz, M. Gruteser, S. Oh, B. Balle, and D. Ramage. Airgapagent: Protecting privacy-conscious conversational agents. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security , pages 3868-3882, 2024. -
  4. [5][5] H. Brown, K. Lee, F. Mireshghallah, R. Shokri, and F. Tramèr. What does it mean for a language model to preserve privacy? Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , 2022. -
  5. [6][6] N. Carlini, F. Tramèr, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. B. Brown, D. X. Song, Ú. Erlingsson, A. Oprea, and C. Raffel. Extracting Training Data from Large Language Models. In USENIX Security Symposium , 2020. -
  6. [7][7] K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. Differentially private empirical risk minimization. Journal of Machine Learning Research , 12(3), 2011. -
  7. [10][10] Z. Cheng, D. Wan, M. Abueg, S. Ghalebikesabi, R. Yi, E. Bagdasarian, B. Balle, S. Mellem, and S. O'Banion. Ci-bench: Benchmarking contextual integrity of ai assistants on synthetic data. arXiv preprint arXiv:2409.13903 , 2024. - · arXiv
  8. [11][11] E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tramèr. Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2024. -
  9. [13][13] M. Duan, A. Suri, N. Mireshghallah, S. Min, W. Shi, L. S. Zettlemoyer, Y. Tsvetkov, Y. Choi, D. Evans, and H. Hajishirzi. Do membership inference attacks work on large language models? ArXiv , abs/2402.07841, 2024. - · arXiv
  10. [14][14] I. Evtimov, A. Zharmagambetov, A. Grattafiori, C. Guo, and K. Chaudhuri. WASP: Benchmarking web agent security against prompt injection attacks. arXiv preprint arXiv:2504.18575 , 2025. - · arXiv
  11. [15][15] S. Ghalebikesabi, E. Bagdasaryan, R. Yi, I. Yona, I. Shumailov, A. Pappu, C. Shi, L. Weidinger, R. Stanforth, L. Berrada, et al. Operationalizing contextual integrity in privacy-conscious assistants. arXiv preprint arXiv:2408.02373 , 2024. - · arXiv
  12. [17][17] Y. He, E. Wang, Y. Rong, Z. Cheng, and H. Chen. Security of ai agents. ArXiv , abs/2406.08689, 2024. - · arXiv
  13. [19][19] J. Y. Koh, R. Lo, L. Jang, V. Duvvur, M. C. Lim, P.-Y. Huang, G. Neubig, S. Zhou, R. Salakhutdinov, and D. Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. ACL , 2024. -
  14. [20][20] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , 2023. -
  15. [23][23] Meta Llama. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 , 2024. - · arXiv
  16. [24][24] N. Mireshghallah, H. Kim, X. Zhou, Y. Tsvetkov, M. Sap, R. Shokri, and Y. Choi. Can llms keep a secret? testing privacy implications of language models via contextual integrity theory. ArXiv , abs/2310.17884, 2023. - · arXiv
  17. [26][26] OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2024. - · arXiv
  18. [27][27] OpenAI. Gpt-4o system card. arXiv preprint arXiv:2410.21276 , 2024. - · arXiv
  19. [28][28] S. G. Patil, T. Zhang, X. Wang, and J. E. Gonzalez. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334 , 2023. - · arXiv
  20. [30][30] Y. Ruan, H. Dong, A. Wang, S. Pitis, Y. Zhou, J. Ba, Y. Dubois, C. J. Maddison, and T. Hashimoto. Identifying the risks of lm agents with an lm-emulated sandbox. In The Twelfth International Conference on Learning Representations , 2024. -
  21. [31][31] Y. Shao, T. Li, W. Shi, Y. Liu, and D. Yang. PrivacyLens: Evaluating privacy norm awareness of language models in action. ArXiv , abs/2409.00138, 2024. - · arXiv
  22. [32][32] R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP) , pages 3-18. IEEE, 2017. -
  23. [33][33] R. Staab, M. Vero, M. Balunovi'c, and M. T. Vechev. Beyond memorization: Violating privacy via inference with large language models. ArXiv , abs/2310.07298, 2023. - · arXiv
  24. [34][34] J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. H. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. CoRR , abs/2201.11903, 2022. - · arXiv
  25. [35][35] F. Wu, N. Zhang, S. Jha, P. D. McDaniel, and C. Xiao. A new era in llm security: Exploring security concerns in real-world llm-based systems. ArXiv , abs/2402.18649, 2024. - · arXiv
  26. [36][36] J. Yang, H. Zhang, F. Li, X. Zou, C. Li, and J. Gao. Set-of-Mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441 , 2023. - · arXiv
  27. [38][38] Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, and Y. Zhang. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing , 4(2): 100211, June 2024. ISSN 2667-2952. doi: 10.1016/j.hcc.2024.100211. - · DOI · Paper page
  28. [41][41] S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, Y. Bisk, D. Fried, U. Alon, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854 , 2023. URL https://webarena.dev . Table 6: Set of possible actions A . Taken from [19] Figure 4: Examples of website representations: accessibility tree (left, taken from Zhou et al. [41]), SOM (right). <!-- image --> · arXiv

Cited by

Sorted by citations

Authors
Zharmagambetov, A., Guo, C., Evtimov, I., Pavlova, M., Salakhutdinov, R., & Chaudhuri, K.
Affiliations
  • University of Waterloo (CA)
  • Carnegie Mellon University (US)
  • University of California San Diego (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
Pages
153145-153170
ISSN
1049-5258 (ISSN-L)
DOI
10.48550/arxiv.2503.09780
arXiv
2503.09780
OpenAlex
W4416033185
Cited by
6
References
0
Citations by year
2026: 6
FWCI
5.69
Type
conference paper
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
privacy leakage, data minimization, autonomous AI agents, sensitive data, GPT-4, Llama 3, Claude

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review