The DOI matches its DataCite record; the references it cites are not checked yet.
Mind2Web: Towards a Generalist Agent for the Web
Open accessAbstract
We introduce Mind2Web, the first dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website. Existing datasets for web agents either use simulated websites or only cover a limited set of websites and tasks, thus not suitable for generalist web agents. With over 2,000 open-ended tasks collected from 137 websites spanning 31 domains and crowdsourced action sequences for the tasks, Mind2Web provides three necessary ingredients for building generalist web agents: 1) diverse domains, websites, and tasks, 2) use of real-world websites instead of simulated and simplified ones, and 3) a broad spectrum of user interaction patterns. Based on Mind2Web, we conduct an initial exploration of using large language models (LLMs) for building generalist web agents. While the raw HTML of real-world websites are often too large to be fed to LLMs, we show that first filtering it with a small LM significantly improves the effectiveness and efficiency of LLMs. Our solution demonstrates a decent level of performance, even on websites or entire domains the model has never seen before, but there is still a substantial room to improve towards truly generalizable agents. We open-source our dataset, model implementation, and trained models (https://osu-nlp-group.github.io/Mind2Web) to facilitate further research on building a generalist agent for the web.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We first run with AWM offline using both GPT variants, and find that AWM consistently obtains the highest success rate in both step and task levels, leading to 4 . 0 -8 . 9 % relative and 0 . 4 -2 . 8 absolute points increases in step-wise and task-wise success rates than the baselines - Synapse with gpt-3.5-turbo and MindAct with gpt-4 .”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As shown in Table 4, both AWM online and AWM offline surpass the MindAct baseline by a large margin, resulting in 7 . 4 -8 . 9 , 3 . 6 -3 . 8 , and 14 . 0 -16 . 9 absolute point improvements in step success rates, in cross-task, cross-website, and cross-domain scenarios.”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In contrast, the second class of benchmarks (Yao et al., 2022; Zhou et al.; Deng et al., 2023), such as SWEBench (Jimenez et al., 2023) and WebArena (Zhou et al.), primarily evaluate action by placing agents in dynamic environments, but are typically confined to a single session.”
From MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks (2026) · cites this paper as Deng et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous works related to web pages focus on addressing action-based requests, such as Mind2Web (Deng et al., 2023) and WebArena (Zhou et al., 2024a); these HTML-based instruction-action benchmarks face challenges such as excessively noisy information and overly long inputs, which can significantly hinder performance due to limitations in long-context understanding.”
From WebWalker: Benchmarking LLMs in Web Traversal (2025) · cites this paper as Deng et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, they fail to capture the complexities of real-world scenarios where relevant information is buried deep within web pages and requires multiple layers of interaction.”
From WebWalker: Benchmarking LLMs in Web Traversal (2025) · cites this paper as Deng et al., 2023
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous benchmarks provide datasets of demonstrations without executable environments [9, 40, 21].”
From OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments (2024) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we also integrate 84 examples from other benchmarks focusing on single-application or domain-specific environments such as NL2Bash [29], Mind2Web [9], SheetCopilot [25], PPTC [14], and GAIA [36].”
From OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments (2024) · cites this paper as [9]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Mind2Web (Deng et al., 2023) features web navigation in cross-task, website, and domain settings, stressing the generality of agents on versatile operations and environments.”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“MindAct introduces webpage element filtering and multi-choice task format to ease observation processing, and Synapse changes the format to a trajectory style and augments retrieved relevant examples.”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Our evaluation data is derived from the Mind2Web dataset (Deng et al., 2023) which is public and cached offline, eliminating the need for attacks on live websites.”
From EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage (2024) · cites this paper as Deng et al., 2023
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Despite more accurate element selection, AWM gets slightly lower action F 1 scores than MindAct, possibly because the augmented workflows may guide the agent to take certain actions aligning to the workflows, which are not always relevant to the particular environment state at hand.”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 11
- Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ B. Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Bryn- jolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen Creel, Jared Quincy Davis, Dorottya Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, and et al. On the opportunities and risks of foundation models. CoRR, abs/2108.07258, 2021. URL https://arxiv.org/abs/2108.07258. · arXiv
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
- Andrea Burns, Deniz Arsan, Sanjna Agrawal, Ranjitha Kumar, Kate Saenko, and Bryan A. Plum- mer. A dataset for interactive vision-language navigation with unknown command feasibility. In European Conference on Computer Vision, 2022.
- Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways, 2023. URL http://jmlr.org/papers/v24/22-1144.html.
- Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: decoding-enhanced bert with disentangled attention. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https: //openreview.net/forum?id=XPZIaotutsD.
- Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. Reinforcement learning on web interfaces using workflow-guided exploration. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 -May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreview.net/ forum?id=ryTp3f-0-.
- Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https://arxiv.org/ abs/1908.10084. · arXiv
- Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang. World of Bits: An Open-Domain Platform for Web-Based Agents. In Proceedings of the 34th International Conference on Machine Learning, pages 3135-3144. PMLR, July 2017.
- Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. ALFRED: A benchmark for interpreting grounded instructions for everyday tasks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 10737-10746.
- Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the- art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online, October 2020. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/ 2020.emnlp-demos.6.
- Nancy Xu, Sam Masling, Michael Du, Giovanni Campagna, Larry Heck, James A. Landay, and Monica S. Lam. Grounding open-domain instructions to automate web support tasks. In North American Chapter of the Association for Computational Linguistics, 2021.
Cited by
Sorted by citations
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
Large Language Models for Information Retrieval: A Survey
Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., & Wen, J.-R. (2026) · ACM Transactions on Information Systems · doi:10.1145/3748304 · Cited by 225
CogAgent: A Visual Language Model for GUI Agents
Hong, W., Wang, W., Lv, Q., Xu, J., Yu, W., Ji, J., Wang, Y., Wang, Z., Dong, Y., Ding, M., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01354 · Cited by 202
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents
Zhan, Q., Liang, Z., Ying, Z., & Kang, D. (2024) · Findings of the Association for Computational Linguistics: ACL 2024 · doi:10.18653/v1/2024.findings-acl.624 · Cited by 147
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 72
“, 2022), web manipulation (Yao et al., 2022a; Deng et al., 2023), puzzle solving (Yao et al.”
BackgroundOSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Xie, T., Zhang, D., Chen, J., Li, X., Zhao, S., Cao, R., Hua, T. J., Cheng, Z., Shin, D., Lei, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.07972 · Cited by 64
Han, S., Wang, M., Zhang, J., Li, D., & Duan, J. (2024) · Electronics · doi:10.3390/electronics13245040 · Cited by 81
Large Language Model Instruction Following: A Survey of Progresses and Challenges
Lou, R., Zhang, K., & Yin, W. (2024) · Computational Linguistics · doi:10.1162/coli_a_00523 · Cited by 133
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2310.06770 · Cited by 72
“, 2023) or turned to the web as an interactive setting featuring tasks that require multiple steps to solve (Yao et al., 2022; Zhou et al., 2023; Deng et al., 2023; Liu et al., 2023d).”
BackgroundLLM-Based Framework for Administrative Task Automation in Healthcare
Gebreab, S. A., Salah, K., Jayaraman, R., Rehman, M. H. U., & Ellaham, S. (2024) · International Symposium on Digital Forensics and Security (ISDFS) · doi:10.1109/isdfs60797.2024.10527275 · Cited by 64
SAGE: Self-evolving Agents with Reflective and Memory-augmented Abilities
Liang, X., Tao, M., Xia, Y., Wang, J., Li, K., Wang, Y., He, Y., Yang, J., Shi, T., Wang, Y., et al. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130470 · Cited by 39
Yuan, Z., Li, M., Liu, C., Han, F., Huang, H., & Dai, H. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.02.008 · Cited by 36
Understanding the planning of LLM agents: A survey
Huang, X., Liu, W., Chen, X., Wang, X., Wang, H., Lian, D., Wang, Y., Tang, R., & Chen, E. (2024) · arXiv · doi:10.48550/arxiv.2402.02716 · Cited by 47
Know where to go: Make LLM a relevant, responsible, and trustworthy searchers
Shi, X. D., Liu, J., Liu, Y., Cheng, Q., & Lu, W. P. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114354 · Cited by 51
Ning, L., Liang, Z., Jiang, Z., Qu, H., Ding, Y., Fan, W., Wei, X.-Y., Lin, S., Liu, H., Yu, P. S., et al. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736555 · Cited by 77
Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
Chhabra, A., Datta, S., Nahin, S. K., & Mohapatra, P. (2026) · IEEE Access · doi:10.1109/access.2026.3675554 · Cited by 28
“Complementary initiatives include the NCCoE Cyber AI Profile [220], which offers implementation guidance, Cybersecurity Framework, for integrating cybersecurity controls into AI systems; the OWASP Agentic AI Threats project [221], which catalogs common vulnerabilities such as insecure orchestration and prompt injectio…”
InfluentialExploring Embodied Multimodal Large Models: Development, datasets, and future directions
Chen, S., Wu, Z., Zhang, K., Li, C., Zhang, B., Ma, F., Yu, F. R., & Li, Q. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103198 · Cited by 39
Self-citationAgentBoard: An analytical evaluation board of multi-turn LLM agents
Ma, C., Zhang, J., Zhu, Z., Yang, C., Yang, Y., Jin, Y., Lan, Z., Kong, L., & He, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-2365 · Cited by 26
Chen, Y., Sikka, K., Cogswell, M., Ji, H., & Divakaran, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01350 · Cited by 56
WebWalker: Benchmarking LLMs in Web Traversal
Wu, J., Yin, W., Jiang, Y., Wang, Z., Xi, Z., Fang, R., Zhang, L., He, Y., Zhou, D., Xie, P., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.508 · Cited by 18
Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions
Gridach, M., Nanavati, J., Abidine, K. Z. E., Mendes, L., & Mack, C. (2025) · arXiv · doi:10.48550/arxiv.2503.08979 · Cited by 18
Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Yin, D., Brahman, F., Ravichander, A., Chandu, K., Chang, K.-W., Choi, Y., & Lin, B. Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.670 · Cited by 17
Sabbagh, V. B., Lima, C. B. D. C., & Xexeo, G. B. (2024) · SPE Journal · doi:10.2118/223612-pa · Cited by 19
AssistGUI: Task-Oriented PC Graphical User Interface Automation
Gao, D., Ji, L., Bai, Z., Ouyang, M., Li, P., Mao, D., Wu, Q., Zhang, W., Wang, P., Guo, X., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01262 · Cited by 73
WebVLN: Vision-and-Language Navigation on Websites
Chen, Q., Pitawela, D., Zhao, C. K., Zhou, G., Chen, H., & Wu, Q. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27878 · Cited by 20
Large Language Models can Strategically Deceive their Users when Put Under Pressure
Scheurer, J., Balesni, M., & Hobbhahn, M. (2023) · arXiv · doi:10.48550/arxiv.2311.07590 · Cited by 11
“While others (Kinniment et al., 2023; OpenAI, 2023; Zhou et al., 2023; Shinn et al., 2023; Deng et al., 2023; Wang et al., 2023; Richards, 2023; Nakajima, 2023; SuperAGI, 2023) give LMs access to real tools, like access to a command line that executes bash commands in a real virtual environment, we…”
AppAgent: Multimodal Agents as Smartphone Users
Zhang, C., Yang, Z., Liu, J., Li, Y., Han, Y., Chen, X., Huang, Z., Fu, B., & Yu, G. (2023) · arXiv · doi:10.48550/arxiv.2312.13771 · Cited by 11
“Recent systems like Mind2Web [8], WebArena [53], WorkArena [9], and VisualWebArena [18] have leveraged LLMs to operate and interact with web pages, while others, such as OSWorld [41], extend similar capabilities to desktop PCs.”
Wang, Z. Z., Mao, J., Fried, D., & Neubig, G. (2024) · arXiv · doi:10.48550/arxiv.2409.07429 · Cited by 5
“Mind2Web (Deng et al., 2023) proposes versatile tasks and stresses agent generalization across websites and domains.”
BackgroundMethodInfluentialAre Autonomous Web Agents Good Testers?
Chevrot, A., Vernotte, A., Falleri, J., Blanc, X., Legeard, B., & Cretin, A. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728879 · Cited by 11
Automated Test Transfer across Android Apps using Large Language Models
Beyzaei, B., Talebipour, S., Rafiei, G., Medvidović, N., & Malek, S. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728975 · Cited by 10
Generative AI Agents for Knowledge Work Augmentation in Finance
Ganesh, S., Ardon, L., Borrajo, D., Garg, D., Madhushani, U., Narayanan, A. L., Canonaco, G., & Veloso, M. (2024) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-022823-033813 · Cited by 13
Eythorsson, D., & Clark, M. (2025) · Hydrological Processes · doi:10.1002/hyp.70065 · Cited by 12
AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
Ma, C., Zhang, J., Zhu, Z., Yang, C., Yang, Y., Jin, Y., Lan, Z., Kong, L., & He, J. (2024) · arXiv · doi:10.48550/arxiv.2401.13178 · Cited by 26
Kapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N., & Narayanan, A. (2024) · arXiv · doi:10.48550/arxiv.2407.01502 · Cited by 13
Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., & Mihalcea, R. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3548 · Cited by 14
“In contrast, more recent efforts evaluate LLMs on complex tasks that resemble real-world applications [18, 38, 75] or involve A/B testing with human feedback [10].”
MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
Huang, Y., Shi, J., Li, Y., Fan, C., Wu, S., Zhang, Q., Liu, Y., Zhou, P., Wan, Y., Gong, N. Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.03128 · Cited by 7
A Zero-Shot Language Agent for Computer Control with Structured Reflection
Li, T., Li, G., Deng, Z., Wang, B. K., & Li, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.753 · Cited by 33
Self-citationAn Evaluation Mechanism of LLM-based Agents on Manipulating APIs
Liu, B., Jianxiang, Z., Meng, D., & Lu, H. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.267 · Cited by 4
Attacking Vision-Language Computer Agents via Pop-ups
Zhang, Y., Yu, T., & Yang, D. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.411 · Cited by 4
Dual-View Visual Contextualization for Web Navigation
Kil, J., Song, C. H., Zheng, B., Deng, X., Su, Y., & Chao, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01369 · Cited by 12
Self-citationR2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memory
Huang, T., Basu, K., Abdelaziz, I., Kapanipathi, P., May, J., & Chen, M. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1464 · Cited by 4
“…research has shown that language models, without intervention, struggle to express linguistic intent in formal instruction that can control an extra-linguistic environment, such as web site navigation (Shi et al., 2017; Liu et al., 2018; Yao et al., 2022; Zhou et al., 2024b; Deng et al., 2023).”
WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?
Drouin, A., Gasse, M., Caccia, M., Laradji, I. H., Del Verme, M., Marty, T., Boisvert, L., Thakkar, M., Cappart, Q., Vazquez, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07718 · Cited by 3
“Other works have also explored web agents that receive HTML as input and produce a series of high-level actions such as click , type , select (Deng et al., 2023; Liu et al., 2023a;b; Yao et al., 2023).”
BackgroundReTA: Recursively Thinking Ahead to Improve the Strategic Reasoning of Large Language Models
Duan, J., Wang, S., Diffenderfer, J., Sun, L., Chen, T., Kailkhura, B., & Xu, K. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.123 · Cited by 7
Visual Test-Time Scaling for GUI Agent Grounding
Luo, T. S., Logeswaran, L., Johnson, J., & Lee, H. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01859 · Cited by 7
Difficulty-Aware Hybrid Routing for Cost-Efficient Long-Context LLM Inference
Gu, Z., Xiao, Z., & Lin, O. (2026) · International Conference on Intelligent Communications and Computing (ICICC) · doi:10.1109/icicc71012.2026.11637729 · Cited by 4
Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
de Witt, C. S., Krawiecka, K., Krawczuk, I., Hagag, B., Anderson, W. L., Belcak, P., Bucknall, B., Cai, X., Chopra, A., Cohen, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.02077 · Cited by 108
“…research (Gottweis et al., 2025; Schmidgall et al., 2025), and negotiating transactions, or using computers (Humphreys et al., 2022; Bonatti et al., 2024) through interfaces originally designed for humans (Shi et al., 2017; Deng et al., 2023; Zhou et al., 2023; Garg et al., 2025; Xue et al., 2025).”
Lu, J., Holleis, T., Zhang, Y., Aumayer, B., Nan, F., Bai, F., Ma, S., Ma, S., Li, M., Yin, G., et al. (2024) · arXiv · doi:10.48550/arxiv.2408.04682 · Cited by 2
“WebArena (Zhou et al., 2023), MiniWoB++ (Humphreys et al., 2022), Webshop (Yao et al., preprint), Mind2Web (Deng et al., 2023) and Visu-alWebArena (Koh et al., 2024) focus on the agent’s ability to call search functionality to browse and leverage the web to solve the task.”
Ying, T., Wang, L., Hu, P., & Ding, T. (2026) · International Conference on Electronic Technology and Information Science (ICETIS) · doi:10.1109/icetis70504.2026.11633437 · Cited by 6
-, S. B., Ranjan, S., Adhitya, M., Suri, V., Hassija, V., Chamola, V., & Hussain, A. (2026) · Cognitive Computation · doi:10.1007/s12559-026-10619-1 · Cited by 5
Analysis of architecture options for foundation-model-based agents: A taxonomy and decision model
Zhou, J., Lu, Q., Chen, J., Zhu, L., Xu, X., Xing, Z., & Harrer, S. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113109 · Cited by 17
RealWeb: A Benchmark for Universal Instruction Following in Realistic Web Services Navigation
Zhang, B., Xiong, S., Sui, D., Xu, Y. D., Tu, Z., & Chu, D. (2024) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws62655.2024.00056 · Cited by 2
EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
Liao, Z., Mo, L., Xu, C., Kang, M., Zhang, J., Xiao, C., Tian, Y., Li, B., & Sun, H. (2024) · arXiv · doi:10.48550/arxiv.2409.11295 · Cited by 1
“Our evaluation data is derived from the Mind2Web dataset (Deng et al., 2023) which is public and cached offline, eliminating the need for attacks on live websites.”
InfluentialSelf-citationLLMAGENTNET: A COLLABORATIVE NETWORK OF AUTONOMOUS AI AGENTS FOR COMPLEX TASK EXECUTION
Бідочко, А., & Виклюк, Я. І. (2025) · Науковий вісник НЛТУ України · doi:10.36930/40350412 · Cited by 3
Knowledge-augmented form-filling agent for higher education services
Sang, H., Liu, M., Tang, J., Yang, Y., & Chen, Y. (2026) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-026-00612-x · Cited by 1
KC-Agent: A Dual-Process Cognitive Architecture for Efficient ML Model Improvement
Bravo-Rocca, G., Guitart, J., Dholakia, A., Ellison, D. D., & Jain, P. K. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00186 · Cited by 1
Mittal, S. (2026) · arXiv · doi:10.48550/arxiv.2609.01660 · Cited by 1
MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
He, Z., Wang, Y., Zhi, C., Hu, Y., Chen, T.-P., Yin, L., Chen, Z., Wu, T. A., Ouyang, S., Wang, Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.16313
“Web-based agent environments such as Web-Shop (Yao et al., 2022), Mind2Web (Deng et al., 2023), and Mind2Web 2 (Gou et al., 2025) assess an agent’s ability to navigate web interfaces, invoke tools, and execute grounded actions in response to web transitions.”
MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
Zhou, Z., Qu, A., Wu, Z., Kim, S., Prakash, A., Rus, D., Zhao, J., Low, B. K. H., & Liang, P. P. (2025) · arXiv · doi:10.48550/arxiv.2506.15841 · Cited by 218
“…of training approaches: (1) behavior cloning (BC), which involves imitating expert trajectories to guide agent behavior by supervised fine-tuning (SFT) [62, 16, 12], and (2) reinforcement learning (RL), which optimizes agent policies by incentivizing desirable outcomes through rewards [47, 5, 39].”
ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
Ouyang, S., Yan, J., Hsu, I.-H., Chen, Y., Jiang, K., Wang, Z., Han, R., Le, L. T., Daruki, S., Tang, X., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.25140 · Cited by 295
“Think first: Why did some trajectories succeed while others failed?”
WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning
Wei, Z., Yao, W., Liu, Y., Zhang, W., Lu, Q., Qiu, L., Yu, C., Xu, P., Zhang, C., Yin, B., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.16421
“Following prior work (Qi et al., 2025), we adopt WebArena (Zhou et al., 2024a) as the web environment over other simulated or static environments such as WebShop (Yao et al., 2022) or Mind2Web (Deng et al., 2023) for greater practicality—”
InfluentialXie, S., Shi, Z., Shen, H., Wang, J., Zhong, S., Liu, M., Pan, C., Jia, P., Sun, B., Jing, X., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.21482
“This taxonomy follows the separation in recent evaluation work between factual retrieval and factuality (43), browsing or web traversal (29; 46; 10), interactive tool use (36; 50), and long-horizon agent tasks (26; 48).”
Useful Memories Become Faulty When Continuously Updated by LLMs
Zhang, D., Lin, Y., Wu, Z., Sun, Y., Li, B., Li, D., & Peng, H. (2026) · arXiv · doi:10.48550/arxiv.2605.12978
Multi-agent Architecture Search via Agentic Supernet
Zhang, G., Niu, L., Fang, J., Wang, K., Bai, L., & Wang, X. (2025) · arXiv · doi:10.48550/arxiv.2502.04180
“…as question answering (Zhu et al., 2024a), data analysis (Hong et al., 2024; Li et al., 2024), code generation (Shinn et al., 2023), web navigation (Deng et al., 2024), and data synthesis (Butt et al., 2024), by equipping LLMs with high-* level features, including persona (Wang et al., 2023b; Chen…”
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents
Wang, Z., Li, Y., Wu, Y., Chen, K., Liu, Z., Wai, F. K., Chen, P.-Y., Thing, V. L. L., Li, B., Tao, D., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.13385
“UnlikeconventionalLLMapplications, these agents perceive content from open web pages and execute actions that directly alter the environment, often with irreversible financial consequences [10, 17, 49].”
UI-Hawk: Unleashing the Screen Stream Understanding for GUI AgentsPreprint
Zhang, J., Yu, Y., Liao, M., Li, W., Wu, J., & Wei, Z. (2024) · Preprints.org · doi:10.20944/preprints202408.2137.v1
Web Agents and Benchmarks - A Survey Focusing on Algorithmic Strategies and Performance RatingsPreprint
Krupp, L., Geißler, D., Lukowicz, P., & Karolus, J. (2025) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/vhn2c_v1
MODELOS DE RACIOCÍNIO E ARQUITETURA EXTENSÍVEL PARA PESQUISA INTELIGENTE
Júnior, H. C. P. (2025) · Congresso Internacional IDEA: Inovação, Diálogo e Experiências na Aprendizagem (IDEA) · doi:10.29327/9786527213314.1086647
The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomes
Gebreegziabher, S. A., Yang, Y., Chiang, C., Yoo, H., Chen, C., Do, H. J., Ashktorab, Z., Geyer, W., Gómez-Zará, D., & Li, T. J.-J. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789219 · Cited by 5
TRUST-CUA: Trustworthy Computer-Using Generalist Agents for Intelligent User Interfaces (Workshop)
Li, T. J.-J., Shlomov, S., Deng, X. B., Brafman, R. I., Yaeli, A., & Wang, Z. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3794960
Self-citationCLMC: Code-Level Metacognition for LLM Agents
Jian, R., Shang, Y., Zhu, Z., Liu, K., Bao, C., & Liu, X. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00295
Training-Free Agentic AI: Probabilistic Control and Coordination in Multi-Agent LLM Systems
Hosseini, M. P., Shah, A., Qureshi, S., Huang, A., Miao, C., & Wei, W. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00034 · Cited by 1
Liu, Y., Shao, S., Guo, S., Wang, C., Yang, C., & Zang, Z. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3680460
Proactive models for open-domain conversational search
Erbacher, P. (2024) · theses.fr (ABES) · doi:10.70675/f7d68861zc694z49f1zb130z675cbf38ed1c
MolmoWeb: Open Visual Web Agent and Open Data for the Open Web
Gupta, T., Wolters, P., Ma, Z., Sushko, P., Pang, R. Y., Yang, Y., Anderson, T., Ren, Z., Trivedi, H., Han, W., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37393-9_17
GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
Ouyang, M., Hu, S., Lin, K. Q., Ng, H. T., & Shou, M. Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37255-0_3 · Cited by 31
"Here, Let Me Help": An Empirical Study of User Interventions in Human–Web Agent Collaboration
Kim, J., Cho, S., Nguyen, D. M., Jeon, J., Shin, M., & Ko, S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791536 · Cited by 1
Specialized Approaches for Visual Instruction Grounding in GUI Automation
Ettifouri, E. H., Alassan, M. S. Y., Bouhandi, M., & Dahhane, W. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-18716-1_6
Audio-Text Jailbreak Attack on Large Audio-Language Models: Towards Generality and Stealthiness
Li, Y., Zhang, J., Chen, Y., Zhao, Z., & Wang, J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464814
PWA: Process-Level Web Agent Reinforcement Learning
Fu, D., Hong, W., He, K., Wang, C., Gongque, Z., Wang, Y., & Xu, W. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11460752
Penna, G. D., Buzzi, M., & Leporini, B. (2026) · Research Square · doi:10.2139/ssrn.6643942
UINavBench: A Framework for Comprehensive Evaluation of Interactive Digital Agents
Agrawal, H., Schoop, E., Pan, X., Mahajan, A., Seff, A., Feng, D., Cheng, R., y Terán, A. R. M., Gomez, E., Sundararajan, A., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02168
See, Record, Do: Automated Generation of UI Workflows from Tutorial Videos
Beauchaine, A. J., & Shue, C. A. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00605
Runtime-Recovery Agent: Solving UI-Related Exceptions in RPA Systems with GUI Agents
Rodríguez-Ruiz, A., Enríquez, J. G., & Ramirez, A. J. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-13426-4_10
Vector: Canonical Intent Graphs and Mode-Aware Execution for State-Persistent Web Interaction
Venkatesan, G., Sharan, B., Rama, P., Kalaivani, J., & Angayarkanni, V. (2026) · International Conference on Electronic Systems and Intelligent Computing (ICESIC) · doi:10.1109/icesic67389.2026.11496552
Singh, A., Mukhopadhyay, A., Saini, M. L., Kumar, A., & Verma, S. K. (2026) · International Conference on Smart Futuristic Technology (SMARTTECH) · doi:10.1109/icsft66733.2026.11506779
Dossier: Deep Research via Ledger-Driven Branching Search and Query Encoding Learning
Chabra, O., Ziems, N., Jiang, M., Khattab, O., & Balakrishnan, H. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813122 · Cited by 1
Real-Time Procedural Learning From Experience for AI Agents
Bi, D., Hu, Y., & Nasir, M. N. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795087 · Cited by 2
Beyond Browser Agents: Distilling Experience for Accessible Web Automation
Yitalo, A., Jovanovic-Hacon, S., Strathman, M., Gill, B., & Khurana, R. (2026) · International Web for All Conference (W4A) · doi:10.1145/3800424.3800431
Adaptive Multi-Source Data Crawling via LLM-Guided Semantic Filtering and Propagation Analysis
Li, H., Cheng, Y., & Luo, S. (2026) · IEEE International Conference on Control, Electronics and Computer Technology (ICCECT) · doi:10.1109/iccect68671.2026.11565337
How Can AI Augment Access to Justice? Public Defenders' Perspectives on Responsible AI Adoption
Cheong, I., Liu, P., Stammbach, D., & Henderson, P. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806526 · Cited by 3
VisCritic: Visual State Comparison as Process Reward for GUI Agents
Qian, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37595-7_36 · Cited by 2
PPAI: Enabling Personalized LLM Agent Interoperability for Collaborative Edge Intelligence
Wang, Z., Liu, Q., Guo, K., Wang, H., Lin, J., Hong, Z., & Guo, S. (2026) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom59046.2026.11571683 · Cited by 1
PlayCoder: Making LLM-Generated GUI Code Playable
Peng, Z., Tao, W., Yin, X., Ying, C., Luo, Y., & Guo, Y. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808097 · Cited by 10
Learning from Failure: Inference-Time Self-improvement for Computer-Use Agents
Sun, X., Wang, X., Schmidt, L., Yeung, S. Y., & Zhang, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37577-3_12 · Cited by 7
WebCloak: Characterizing and Mitigating Threats From LLM-Driven Web Agents as Intelligent Scrapers
Li, X., Qiu, T., Jin, Y., Wang, L., Guo, H., Jia, X., Wang, X., & Dong, W. (2026) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp63933.2026.00025
Persona Diversity: Comparing Usability-Testing Outcomes Across Heterogeneous Personas
Baek, Y., Tozer, C., Ghimire, A., & Nizamani, S. B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30846-7_16
Liu, Y., Zhu, W., & Fan, R. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11581472
WebRetriever: A Large-Scale Comprehensive Benchmark for Efficient Web Agent Evaluation
Dong, W., Fu, T., Yu, Z., Wang, H., Su, A., Fang, Z., Chen, Y., Wang, S., Wu, M., Jiang, P., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37035-8_28
Beyond Fluency: Toward Reliable Trajectories in Agentic IR
Sinha, A., Ranganathan, S., Das, D., & Dharmaratnakar, A. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808511 · Cited by 3
Metadata
- Authors
- Deng, X., Gu, Y., Zheng, B., Chen, S., Stevens, S., Wang, B., Sun, H., & Su, Y.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 28091-28114
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2306.06070
- OpenAlex
- W7133215859
- Cited by
- 134
- References
- 0
- Citations by year
- 2026: 104 · 2025: 14 · 2024: 15 · 2023: 1
- FWCI
- 34.44
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1942980, 2112606, CAREER)
- Topics
- Physical Sciences › Computer Science › Information Systems › Web Data Mining and Analysis
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Keywords
- web agents, web search, large language models, information retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
