Mind2Web: Towards a Generalist Agent for the Web
Open accessAbstract
We introduce Mind2Web, the first dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website. Existing datasets for web agents either use simulated websites or only cover a limited set of websites and tasks, thus not suitable for generalist web agents. With over 2,000 open-ended tasks collected from 137 websites spanning 31 domains and crowdsourced action sequences for the tasks, Mind2Web provides three necessary ingredients for building generalist web agents: 1) diverse domains, websites, and tasks, 2) use of real-world websites instead of simulated and simplified ones, and 3) a broad spectrum of user interaction patterns. Based on Mind2Web, we conduct an initial exploration of using large language models (LLMs) for building generalist web agents. While the raw HTML of real-world websites are often too large to be fed to LLMs, we show that first filtering it with a small LM significantly improves the effectiveness and efficiency of LLMs. Our solution demonstrates a decent level of performance, even on websites or entire domains the model has never seen before, but there is still a substantial room to improve towards truly generalizable agents. We open-source our dataset, model implementation, and trained models (https://osu-nlp-group.github.io/Mind2Web) to facilitate further research on building a generalist agent for the web.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AWM offline on Mind2Web cross-task dataset consistently obtains the highest success rate in both step and task levels, leading to 4.0-8.9% relative and 0.4-2.8 absolute points increases in step-wise and task-wise success rates than the baselines.
“We first run with AWM offline using both GPT variants, and find that AWM consistently obtains the highest success rate in both step and task levels, leading to 4 . 0 -8 . 9 % relative and 0 . 4 -2 . 8 absolute points increases in step-wise and task-wise success rates than the baselines - Synapse with gpt-3.5-turbo and MindAct with gpt-4 .”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
Section 4.1 describes Mind2Web’s Cross-Task split, and Section 4.2 defines step and task success rates. The cited paper does not report AWM offline results or the claimed improvements over Synapse and MindAct; those are results stated by the citing authors, not established by this source. (LLM, conf=0.87)
Experimental — accuracy not yet measuredAWM online and AWM offline surpass the MindAct baseline by a large margin on Mind2Web, resulting in 7.4-8.9, 3.6-3.8, and 14.0-16.9 absolute point improvements in step success rates, in cross-task, cross-website, and cross-domain scenarios.
“As shown in Table 4, both AWM online and AWM offline surpass the MindAct baseline by a large margin, resulting in 7 . 4 -8 . 9 , 3 . 6 -3 . 8 , and 14 . 0 -16 . 9 absolute point improvements in step success rates, in cross-task, cross-website, and cross-domain scenarios.”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
Section 3 introduces MINDACT, and Table 2 reports its step success rates on Mind2Web’s cross-task, cross-website, and cross-domain splits. The source does not report AWM online or offline results or the stated 7.4–8.9, 3.6–3.8, and 14.0–16.9 point improvements over MINDACT. (LLM, conf=0.88)
Experimental — accuracy not yet measuredThe second class of benchmarks primarily evaluate action by placing agents in dynamic environments, but are typically confined to a single session.
“In contrast, the second class of benchmarks (Yao et al., 2022; Zhou et al.; Deng et al., 2023), such as SWEBench (Jimenez et al., 2023) and WebArena (Zhou et al.), primarily evaluate action by placing agents in dynamic environments, but are typically confined to a single session.”
From MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks (2026) · cites this paper as Deng et al., 2023
The Introduction and Section 3 describe web tasks involving sequences of actions, supporting the claim that Mind2Web evaluates agent action. But “Human-Agent Interaction” says evaluation uses cached offline snapshots rather than live dynamic environments, and the passages do not establish that evaluation is confined to a single session. (LLM, conf=0.86)
Experimental — accuracy not yet measuredPrevious HTML-based instruction-action benchmarks face challenges such as excessively noisy information and overly long inputs, which can significantly hinder performance due to limitations in long-context understanding.
“Previous works related to web pages focus on addressing action-based requests, such as Mind2Web (Deng et al., 2023) and WebArena (Zhou et al., 2024a); these HTML-based instruction-action benchmarks face challenges such as excessively noisy information and overly long inputs, which can significantly hinder performance due to limitations in long-context understanding.”
From WebWalker: Benchmarking LLMs in Web Traversal (2025) · cites this paper as Deng et al., 2023
Section 2.3 describes Mind2Web as an action-based web benchmark and says its real-world pages contain long, complex DOM structures that challenge modeling. Sections 4.2 and 3 describe cleaning HTML and say feeding raw pages directly to LLMs can be infeasible or costly. The paper does not establish that noisy or long inputs significantly hinder performance specifically because of limitations in long-context understanding. (LLM, conf=0.84)
Experimental — accuracy not yet measuredPrevious HTML-based instruction-action benchmarks fail to capture the complexities of real-world scenarios where relevant information is buried deep within web pages and requires multiple layers of interaction.
“Additionally, they fail to capture the complexities of real-world scenarios where relevant information is buried deep within web pages and requires multiple layers of interaction.”
From WebWalker: Benchmarking LLMs in Web Traversal (2025) · cites this paper as Deng et al., 2023
Section 2.3, “Comparison with Existing Work and Research Challenges” (page unavailable), says earlier studies used simpler environments and step-by-step directives, whereas real-world pages have complex DOM structures and require greater planning. This supports the comparison with previous benchmarks, but the copy does not specifically say that relevant information is buried deep within pages or requires multiple layers of interaction. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Mind2Web features web navigation in cross-task, website, and domain settings, stressing the generality of agents on versatile operations and environments.
“Mind2Web (Deng et al., 2023) features web navigation in cross-task, website, and domain settings, stressing the generality of agents on versatile operations and environments.”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
MindAct introduces webpage element filtering and multi-choice task format to ease observation processing.
“MindAct introduces webpage element filtering and multi-choice task format to ease observation processing, and Synapse changes the format to a trajectory style and augments retrieved relevant examples.”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
The evaluation data is derived from the Mind2Web dataset, which is public and cached offline, eliminating the need for attacks on live websites.
“Our evaluation data is derived from the Mind2Web dataset (Deng et al., 2023) which is public and cached offline, eliminating the need for attacks on live websites.”
From EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage (2024) · cites this paper as Deng et al., 2023
AWM gets slightly lower action F1 scores than MindAct on Mind2Web cross-task.
“Despite more accurate element selection, AWM gets slightly lower action F 1 scores than MindAct, possibly because the augmented workflows may guide the agent to take certain actions aligning to the workflows, which are not always relevant to the particular environment state at hand.”
From Agent Workflow Memory (2024) · cites this paper as Deng et al., 2023
Table 2 reports MINDACT’s Cross-Task operation F1 scores, but it does not report AWM results. Searches for AWM action F1 and augmented workflows found no comparison with MINDACT, so the claimed difference cannot be checked against this paper. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 11
- Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ B. Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Bryn- jolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen Creel, Jared Quincy Davis, Dorottya Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, and et al. On the opportunities and risks of foundation models. CoRR, abs/2108.07258, 2021. URL https://arxiv.org/abs/2108.07258. · arXiv
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
- Andrea Burns, Deniz Arsan, Sanjna Agrawal, Ranjitha Kumar, Kate Saenko, and Bryan A. Plum- mer. A dataset for interactive vision-language navigation with unknown command feasibility. In European Conference on Computer Vision, 2022.
- Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways, 2023. URL http://jmlr.org/papers/v24/22-1144.html.
- Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: decoding-enhanced bert with disentangled attention. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https: //openreview.net/forum?id=XPZIaotutsD.
- Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. Reinforcement learning on web interfaces using workflow-guided exploration. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 -May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreview.net/ forum?id=ryTp3f-0-.
- Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https://arxiv.org/ abs/1908.10084. · arXiv
- Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang. World of Bits: An Open-Domain Platform for Web-Based Agents. In Proceedings of the 34th International Conference on Machine Learning, pages 3135-3144. PMLR, July 2017.
- Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. ALFRED: A benchmark for interpreting grounded instructions for everyday tasks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 10737-10746.
- Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the- art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online, October 2020. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/ 2020.emnlp-demos.6.
- Nancy Xu, Sam Masling, Michael Du, Giovanni Campagna, Larry Heck, James A. Landay, and Monica S. Lam. Grounding open-domain instructions to automate web support tasks. In North American Chapter of the Association for Computational Linguistics, 2021.
Cited by
Sorted by citations
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970 · see citing works
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · see citing works
CogAgent: A Visual Language Model for GUI Agents
Hong, W., Wang, W., Lv, Q., Xu, J., Yu, W., Ji, J., Wang, Y., Wang, Z., Dong, Y., Ding, M., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01354 · Cited by 202 · see citing works
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents
Zhan, Q., Liang, Z., Ying, Z., & Kang, D. (2024) · Findings of the Association for Computational Linguistics: ACL 2024 · doi:10.18653/v1/2024.findings-acl.624 · Cited by 147 · see citing works
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 72 · see citing works
“, 2022), web manipulation (Yao et al., 2022a; Deng et al., 2023), puzzle solving (Yao et al.”
BackgroundHan, S., Wang, M., Zhang, J., Li, D., & Duan, J. (2024) · Electronics · doi:10.3390/electronics13245040 · Cited by 72 · see citing works
Large Language Model Instruction Following: A Survey of Progresses and Challenges
Lou, R., Zhang, K., & Yin, W. (2023) · Computational Linguistics · doi:10.1162/coli_a_00523 · Cited by 74 · see citing works
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2310.06770 · Cited by 72 · see citing works
“, 2023) or turned to the web as an interactive setting featuring tasks that require multiple steps to solve (Yao et al., 2022; Zhou et al., 2023; Deng et al., 2023; Liu et al., 2023d).”
BackgroundLLM-Based Framework for Administrative Task Automation in Healthcare
Gebreab, S. A., Salah, K., Jayaraman, R., Rehman, M. H. U., & Ellaham, S. (2024) · International Symposium on Digital Forensics and Security (ISDFS) · doi:10.1109/isdfs60797.2024.10527275 · Cited by 56 · see citing works
SAGE: Self-evolving Agents with Reflective and Memory-augmented Abilities
Liang, X., Tao, M., Xia, Y., Wang, J., Li, K., Wang, Y., He, Y., Yang, J., Shi, T., Wang, Y., et al. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130470 · Cited by 39 · see citing works
Yuan, Z., Li, M., Liu, C., Han, F., Huang, H., & Dai, H. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.02.008 · Cited by 36 · see citing works
Understanding the planning of LLM agents: A survey
Huang, X., Liu, W., Chen, X., Wang, X., Wang, H., Lian, D., Wang, Y., Tang, R., & Chen, E. (2024) · arXiv · doi:10.48550/arxiv.2402.02716 · Cited by 47 · see citing works
Know where to go: Make LLM a relevant, responsible, and trustworthy searchers
Shi, X. D., Liu, J., Liu, Y., Cheng, Q., & Lu, W. P. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114354 · Cited by 31 · see citing works
Ning, L., Liang, Z., Jiang, Z., Qu, H., Ding, Y., Fan, W., Wei, X.-Y., Lin, S., Liu, H., Yu, P. S., et al. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736555 · Cited by 41 · see citing works
Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
Chhabra, A., Datta, S., Nahin, S. K., & Mohapatra, P. (2026) · IEEE Access · doi:10.1109/access.2026.3675554 · Cited by 28 · see citing works
“Complementary initiatives include the NCCoE Cyber AI Profile [220], which offers implementation guidance, Cybersecurity Framework, for integrating cybersecurity controls into AI systems; the OWASP Agentic AI Threats project [221], which catalogs common vulnerabilities such as insecure orchestration and prompt injectio…”
InfluentialExploring Embodied Multimodal Large Models: Development, datasets, and future directions
Chen, S., Wu, Z., Zhang, K., Li, C., Zhang, B., Ma, F., Yu, F. R., & Li, Q. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103198 · Cited by 31 · see citing works
Self-citationAgentBoard: An analytical evaluation board of multi-turn LLM agents
Ma, C., Zhang, J., Zhu, Z., Yang, C., Yang, Y., Jin, Y., Lan, Z., Kong, L., & He, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-2365 · Cited by 26 · see citing works
Chen, Y., Sikka, K., Cogswell, M., Ji, H., & Divakaran, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01350 · Cited by 29 · see citing works
WebWalker: Benchmarking LLMs in Web Traversal
Wu, J., Yin, W., Jiang, Y., Wang, Z., Xi, Z., Fang, R., Zhang, L., He, Y., Zhou, D., Xie, P., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.508 · Cited by 18 · see citing works
Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions
Gridach, M., Nanavati, J., Abidine, K. Z. E., Mendes, L., & Mack, C. (2025) · arXiv · doi:10.48550/arxiv.2503.08979 · Cited by 18 · see citing works
Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Yin, D., Brahman, F., Ravichander, A., Chandu, K., Chang, K.-W., Choi, Y., & Lin, B. Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.670 · Cited by 17 · see citing works
Sabbagh, V. B., Lima, C. B. D. C., & Xexeo, G. B. (2024) · SPE Journal · doi:10.2118/223612-pa · Cited by 16 · see citing works
AssistGUI: Task-Oriented PC Graphical User Interface Automation
Gao, D., Ji, L., Bai, Z., Ouyang, M., Li, P., Mao, D., Wu, Q., Zhang, W., Wang, P., Guo, X., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01262 · Cited by 14 · see citing works
WebVLN: Vision-and-Language Navigation on Websites
Chen, Q., Pitawela, D., Zhao, C. K., Zhou, G., Chen, H., & Wu, Q. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27878 · Cited by 13 · see citing works
Large Language Models can Strategically Deceive their Users when Put Under Pressure
Scheurer, J., Balesni, M., & Hobbhahn, M. (2023) · arXiv · doi:10.48550/arxiv.2311.07590 · Cited by 11 · see citing works
“While others (Kinniment et al., 2023; OpenAI, 2023; Zhou et al., 2023; Shinn et al., 2023; Deng et al., 2023; Wang et al., 2023; Richards, 2023; Nakajima, 2023; SuperAGI, 2023) give LMs access to real tools, like access to a command line that executes bash commands in a real virtual environment, we…”
AppAgent: Multimodal Agents as Smartphone Users
Zhang, C., Yang, Z., Liu, J., Li, Y., Han, Y., Chen, X., Huang, Z., Fu, B., & Yu, G. (2023) · arXiv · doi:10.48550/arxiv.2312.13771 · Cited by 11 · see citing works
“Recent systems like Mind2Web [8], WebArena [53], WorkArena [9], and VisualWebArena [18] have leveraged LLMs to operate and interact with web pages, while others, such as OSWorld [41], extend similar capabilities to desktop PCs.”
Wang, Z. Z., Mao, J., Fried, D., & Neubig, G. (2024) · arXiv · doi:10.48550/arxiv.2409.07429 · Cited by 5 · see citing works
“Mind2Web (Deng et al., 2023) proposes versatile tasks and stresses agent generalization across websites and domains.”
BackgroundMethodInfluentialAre Autonomous Web Agents Good Testers?
Chevrot, A., Vernotte, A., Falleri, J., Blanc, X., Legeard, B., & Cretin, A. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728879 · Cited by 11 · see citing works
Automated Test Transfer across Android Apps using Large Language Models
Beyzaei, B., Talebipour, S., Rafiei, G., Medvidović, N., & Malek, S. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728975 · Cited by 10 · see citing works
Generative AI Agents for Knowledge Work Augmentation in Finance
Ganesh, S., Ardon, L., Borrajo, D., Garg, D., Madhushani, U., Narayanan, A. L., Canonaco, G., & Veloso, M. (2024) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-022823-033813 · Cited by 10 · see citing works
Eythorsson, D., & Clark, M. (2025) · Hydrological Processes · doi:10.1002/hyp.70065 · Cited by 10 · see citing works
AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
Ma, C., Zhang, J., Zhu, Z., Yang, C., Yang, Y., Jin, Y., Lan, Z., Kong, L., & He, J. (2024) · arXiv · doi:10.48550/arxiv.2401.13178 · Cited by 26 · see citing works
Kapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N., & Narayanan, A. (2024) · arXiv · doi:10.48550/arxiv.2407.01502 · Cited by 13 · see citing works
Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., & Mihalcea, R. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3548 · Cited by 14 · see citing works
“In contrast, more recent efforts evaluate LLMs on complex tasks that resemble real-world applications [18, 38, 75] or involve A/B testing with human feedback [10].”
MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
Huang, Y., Shi, J., Li, Y., Fan, C., Wu, S., Zhang, Q., Liu, Y., Zhou, P., Wan, Y., Gong, N. Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.03128 · Cited by 7 · see citing works
A Zero-Shot Language Agent for Computer Control with Structured Reflection
Li, T., Li, G., Deng, Z., Wang, B. K., & Li, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.753 · Cited by 10 · see citing works
Self-citationAn Evaluation Mechanism of LLM-based Agents on Manipulating APIs
Liu, B., Jianxiang, Z., Meng, D., & Lu, H. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.267 · Cited by 4 · see citing works
Attacking Vision-Language Computer Agents via Pop-ups
Zhang, Y., Yu, T., & Yang, D. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.411 · Cited by 4 · see citing works
Dual-View Visual Contextualization for Web Navigation
Kil, J., Song, C. H., Zheng, B., Deng, X., Su, Y., & Chao, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01369 · Cited by 5 · see citing works
Self-citationR2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memory
Huang, T., Basu, K., Abdelaziz, I., Kapanipathi, P., May, J., & Chen, M. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1464 · Cited by 4 · see citing works
“…research has shown that language models, without intervention, struggle to express linguistic intent in formal instruction that can control an extra-linguistic environment, such as web site navigation (Shi et al., 2017; Liu et al., 2018; Yao et al., 2022; Zhou et al., 2024b; Deng et al., 2023).”
WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?
Drouin, A., Gasse, M., Caccia, M., Laradji, I. H., Del Verme, M., Marty, T., Boisvert, L., Thakkar, M., Cappart, Q., Vazquez, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07718 · Cited by 3 · see citing works
“Other works have also explored web agents that receive HTML as input and produce a series of high-level actions such as click , type , select (Deng et al., 2023; Liu et al., 2023a;b; Yao et al., 2023).”
BackgroundReTA: Recursively Thinking Ahead to Improve the Strategic Reasoning of Large Language Models
Duan, J., Wang, S., Diffenderfer, J., Sun, L., Chen, T., Kailkhura, B., & Xu, K. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.123 · Cited by 3 · see citing works
Visual Test-Time Scaling for GUI Agent Grounding
Luo, T. S., Logeswaran, L., Johnson, J., & Lee, H. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01859 · Cited by 7 · see citing works
Difficulty-Aware Hybrid Routing for Cost-Efficient Long-Context LLM Inference
Gu, Z., Xiao, Z., & Lin, O. (2026) · International Conference on Intelligent Communications and Computing (ICICC) · doi:10.1109/icicc71012.2026.11637729 · Cited by 3 · see citing works
Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
de Witt, C. S., Krawiecka, K., Krawczuk, I., Hagag, B., Anderson, W. L., Belcak, P., Bucknall, B., Cai, X., Chopra, A., Cohen, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.02077 · Cited by 2 · see citing works
“…research (Gottweis et al., 2025; Schmidgall et al., 2025), and negotiating transactions, or using computers (Humphreys et al., 2022; Bonatti et al., 2024) through interfaces originally designed for humans (Shi et al., 2017; Deng et al., 2023; Zhou et al., 2023; Garg et al., 2025; Xue et al., 2025).”
Lu, J., Holleis, T., Zhang, Y., Aumayer, B., Nan, F., Bai, F., Ma, S., Ma, S., Li, M., Yin, G., et al. (2024) · arXiv · doi:10.48550/arxiv.2408.04682 · Cited by 2 · see citing works
Ying, T., Wang, L., Hu, P., & Ding, T. (2026) · International Conference on Electronic Technology and Information Science (ICETIS) · doi:10.1109/icetis70504.2026.11633437 · Cited by 5 · see citing works
-, S. B., Ranjan, S., Adhitya, M., Suri, V., Hassija, V., Chamola, V., & Hussain, A. (2026) · Cognitive Computation · doi:10.1007/s12559-026-10619-1 · Cited by 3 · see citing works
Analysis of architecture options for foundation-model-based agents: A taxonomy and decision model
Zhou, J., Lu, Q., Chen, J., Zhu, L., Xu, X., Xing, Z., & Harrer, S. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113109 · Cited by 2 · see citing works
RealWeb: A Benchmark for Universal Instruction Following in Realistic Web Services Navigation
Zhang, B., Xiong, S., Sui, D., Xu, Y. D., Tu, Z., & Chu, D. (2024) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws62655.2024.00056 · Cited by 2 · see citing works
EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
Liao, Z., Mo, L., Xu, C., Kang, M., Zhang, J., Xiao, C., Tian, Y., Li, B., & Sun, H. (2024) · arXiv · doi:10.48550/arxiv.2409.11295 · Cited by 1 · see citing works
“Our evaluation data is derived from the Mind2Web dataset (Deng et al., 2023) which is public and cached offline, eliminating the need for attacks on live websites.”
InfluentialSelf-citationLLMAGENTNET: A COLLABORATIVE NETWORK OF AUTONOMOUS AI AGENTS FOR COMPLEX TASK EXECUTION
Бідочко, А., & Виклюк, Я. І. (2025) · Науковий вісник НЛТУ України · doi:10.36930/40350412 · Cited by 1 · see citing works
Knowledge-augmented form-filling agent for higher education services
Sang, H., Liu, M., Tang, J., Yang, Y., & Chen, Y. (2026) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-026-00612-x · Cited by 1 · see citing works
KC-Agent: A Dual-Process Cognitive Architecture for Efficient ML Model Improvement
Bravo-Rocca, G., Guitart, J., Dholakia, A., Ellison, D. D., & Jain, P. K. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00186 · Cited by 1 · see citing works
MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
He, Z., Wang, Y., Zhi, C., Hu, Y., Chen, T.-P., Yin, L., Chen, Z., Wu, T. A., Ouyang, S., Wang, Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.16313
“Web-based agent environments such as Web-Shop (Yao et al., 2022), Mind2Web (Deng et al., 2023), and Mind2Web 2 (Gou et al., 2025) assess an agent’s ability to navigate web interfaces, invoke tools, and execute grounded actions in response to web transitions.”
MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
Zhou, Z., Qu, A., Wu, Z., Kim, S., Prakash, A., Rus, D., Zhao, J., Low, B. K. H., & Liang, P. P. (2025) · arXiv · doi:10.48550/arxiv.2506.15841
“…of training approaches: (1) behavior cloning (BC), which involves imitating expert trajectories to guide agent behavior by supervised fine-tuning (SFT) [62, 16, 12], and (2) reinforcement learning (RL), which optimizes agent policies by incentivizing desirable outcomes through rewards [47, 5, 39].”
ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
Ouyang, S., Yan, J., Hsu, I.-H., Chen, Y., Jiang, K., Wang, Z., Han, R., Le, L. T., Daruki, S., Tang, X., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.25140
“Think first: Why did some trajectories succeed while others failed?”
WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning
Wei, Z., Yao, W., Liu, Y., Zhang, W., Lu, Q., Qiu, L., Yu, C., Xu, P., Zhang, C., Yin, B., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.16421
“Following prior work (Qi et al., 2025), we adopt WebArena (Zhou et al., 2024a) as the web environment over other simulated or static environments such as WebShop (Yao et al., 2022) or Mind2Web (Deng et al., 2023) for greater practicality—”
InfluentialXie, S., Shi, Z., Shen, H., Wang, J., Zhong, S., Liu, M., Pan, C., Jia, P., Sun, B., Jing, X., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.21482
“This taxonomy follows the separation in recent evaluation work between factual retrieval and factuality (43), browsing or web traversal (29; 46; 10), interactive tool use (36; 50), and long-horizon agent tasks (26; 48).”
Useful Memories Become Faulty When Continuously Updated by LLMs
Zhang, D., Lin, Y., Wu, Z., Sun, Y., Li, B., Li, D., & Peng, H. (2026) · arXiv · doi:10.48550/arxiv.2605.12978
Multi-agent Architecture Search via Agentic Supernet
Zhang, G., Niu, L., Fang, J., Wang, K., Bai, L., & Wang, X. (2025) · arXiv · doi:10.48550/arxiv.2502.04180
“…as question answering (Zhu et al., 2024a), data analysis (Hong et al., 2024; Li et al., 2024), code generation (Shinn et al., 2023), web navigation (Deng et al., 2024), and data synthesis (Butt et al., 2024), by equipping LLMs with high-* level features, including persona (Wang et al., 2023b; Chen…”
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents
Wang, Z., Li, Y., Wu, Y., Chen, K., Liu, Z., Wai, F. K., Chen, P.-Y., Thing, V. L. L., Li, B., Tao, D., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.13385
“UnlikeconventionalLLMapplications, these agents perceive content from open web pages and execute actions that directly alter the environment, often with irreversible financial consequences [10, 17, 49].”
UI-Hawk: Unleashing the Screen Stream Understanding for GUI AgentsPreprint
Zhang, J., Yu, Y., Liao, M., Li, W., Wu, J., & Wei, Z. (2024) · Preprints.org · doi:10.20944/preprints202408.2137.v1
Web Agents and Benchmarks - A Survey Focusing on Algorithmic Strategies and Performance RatingsPreprint
Krupp, L., Geißler, D., Lukowicz, P., & Karolus, J. (2025) · doi:10.31219/osf.io/vhn2c_v1
MODELOS DE RACIOCÍNIO E ARQUITETURA EXTENSÍVEL PARA PESQUISA INTELIGENTE
Júnior, H. C. P. (2025) · Congresso Internacional IDEA: Inovação, Diálogo e Experiências na Aprendizagem (IDEA) · doi:10.29327/9786527213314.1086647
The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomes
Gebreegziabher, S. A., Yang, Y., Chiang, C., Yoo, H., Chen, C., Do, H. J., Ashktorab, Z., Geyer, W., Gómez-Zará, D., & Li, T. J.-J. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789219
TRUST-CUA: Trustworthy Computer-Using Generalist Agents for Intelligent User Interfaces (Workshop)
Li, T. J.-J., Shlomov, S., Deng, X. B., Brafman, R. I., Yaeli, A., & Wang, Z. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3794960
Self-citationCLMC: Code-Level Metacognition for LLM Agents
Jian, R., Shang, Y., Zhu, Z., Liu, K., Bao, C., & Liu, X. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00295
Training-Free Agentic AI: Probabilistic Control and Coordination in Multi-Agent LLM Systems
Hosseini, M. P., Shah, A., Qureshi, S., Huang, A., Miao, C., & Wei, W. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00034
Liu, Y., Shao, S., Guo, S., Wang, C., Yang, C., & Zang, Z. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3680460
Proactive models for open-domain conversational search
Erbacher, P. (2024) · theses.fr (ABES) · doi:10.70675/f7d68861zc694z49f1zb130z675cbf38ed1c
MolmoWeb: Open Visual Web Agent and Open Data for the Open Web
Gupta, T., Wolters, P., Ma, Z., Sushko, P., Pang, R. Y., Yang, Y., Anderson, T., Ren, Z., Trivedi, H., Han, W., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37393-9_17
GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
Ouyang, M., Hu, S., Lin, K. Q., Ng, H. T., & Shou, M. Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37255-0_3
"Here, Let Me Help": An Empirical Study of User Interventions in Human–Web Agent Collaboration
Kim, J., Cho, S., Nguyen, D. M., Jeon, J., Shin, M., & Ko, S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791536
Specialized Approaches for Visual Instruction Grounding in GUI Automation
Ettifouri, E. H., Alassan, M. S. Y., Bouhandi, M., & Dahhane, W. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-18716-1_6
Audio-Text Jailbreak Attack on Large Audio-Language Models: Towards Generality and Stealthiness
Li, Y., Zhang, J., Chen, Y., Zhao, Z., & Wang, J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464814
PWA: Process-Level Web Agent Reinforcement Learning
Fu, D., Hong, W., He, K., Wang, C., Gongque, Z., Wang, Y., & Xu, W. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11460752
Penna, G. D., Buzzi, M., & Leporini, B. (2026) · Research Square · doi:10.2139/ssrn.6643942
UINavBench: A Framework for Comprehensive Evaluation of Interactive Digital Agents
Agrawal, H., Schoop, E., Pan, X., Mahajan, A., Seff, A., Feng, D., Cheng, R., y Terán, A. R. M., Gomez, E., Sundararajan, A., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02168
See, Record, Do: Automated Generation of UI Workflows from Tutorial Videos
Beauchaine, A. J., & Shue, C. A. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00605
Runtime-Recovery Agent: Solving UI-Related Exceptions in RPA Systems with GUI Agents
Rodríguez-Ruiz, A., Enríquez, J. G., & Ramirez, A. J. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-13426-4_10
Vector: Canonical Intent Graphs and Mode-Aware Execution for State-Persistent Web Interaction
Venkatesan, G., Sharan, B., Rama, P., Kalaivani, J., & Angayarkanni, V. (2026) · International Conference on Electronic Systems and Intelligent Computing (ICESIC) · doi:10.1109/icesic67389.2026.11496552
Singh, A., Mukhopadhyay, A., Saini, M. L., Kumar, A., & Verma, S. K. (2026) · International Conference on Smart Futuristic Technology (SMARTTECH) · doi:10.1109/icsft66733.2026.11506779
Dossier: Deep Research via Ledger-Driven Branching Search and Query Encoding Learning
Chabra, O., Ziems, N., Jiang, M., Khattab, O., & Balakrishnan, H. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813122
Real-Time Procedural Learning From Experience for AI Agents
Bi, D., Hu, Y., & Nasir, M. N. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795087
Beyond Browser Agents: Distilling Experience for Accessible Web Automation
Yitalo, A., Jovanovic-Hacon, S., Strathman, M., Gill, B., & Khurana, R. (2026) · International Web for All Conference (W4A) · doi:10.1145/3800424.3800431
Adaptive Multi-Source Data Crawling via LLM-Guided Semantic Filtering and Propagation Analysis
Li, H., Cheng, Y., & Luo, S. (2026) · IEEE International Conference on Control, Electronics and Computer Technology (ICCECT) · doi:10.1109/iccect68671.2026.11565337
How Can AI Augment Access to Justice? Public Defenders' Perspectives on Responsible AI Adoption
Cheong, I., Liu, P., Stammbach, D., & Henderson, P. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806526 · Cited by 2 · see citing works
VisCritic: Visual State Comparison as Process Reward for GUI Agents
Qian, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37595-7_36
PPAI: Enabling Personalized LLM Agent Interoperability for Collaborative Edge Intelligence
Wang, Z., Liu, Q., Guo, K., Wang, H., Lin, J., Hong, Z., & Guo, S. (2026) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom59046.2026.11571683
PlayCoder: Making LLM-Generated GUI Code Playable
Peng, Z., Tao, W., Yin, X., Ying, C., Luo, Y., & Guo, Y. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808097 · Cited by 2 · see citing works
Learning from Failure: Inference-Time Self-improvement for Computer-Use Agents
Sun, X., Wang, X., Schmidt, L., Yeung, S. Y., & Zhang, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37577-3_12
WebCloak: Characterizing and Mitigating Threats From LLM-Driven Web Agents as Intelligent Scrapers
Li, X., Qiu, T., Jin, Y., Wang, L., Guo, H., Jia, X., Wang, X., & Dong, W. (2026) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp63933.2026.00025
Persona Diversity: Comparing Usability-Testing Outcomes Across Heterogeneous Personas
Baek, Y., Tozer, C., Ghimire, A., & Nizamani, S. B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30846-7_16
Liu, Y., Zhu, W., & Fan, R. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11581472
WebRetriever: A Large-Scale Comprehensive Benchmark for Efficient Web Agent Evaluation
Dong, W., Fu, T., Yu, Z., Wang, H., Su, A., Fang, Z., Chen, Y., Wang, S., Wu, M., Jiang, P., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37035-8_28
Beyond Fluency: Toward Reliable Trajectories in Agentic IR
Sinha, A., Ranganathan, S., Das, D., & Dharmaratnakar, A. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808511
Task-Adaptive Retrieval over Agentic Multi-Modal Web Histories via Learned Graph Memory
Forouzandeh, S., Berahmand, K., & Jalili, M. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809978
MemoPlan: Long-Horizon Web Agents with Differential Memory Compression and Lookahead Planning
Li, M., Zhu, R., & Liu, C. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3438-7_42
CoRa-Web: Hierarchical Visual Planning for Robust and Efficient Web Agents
Wang, Y., Wu, J., & Li, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3438-7_48
- Authors
- Deng, X., Gu, Y., Zheng, B., Chen, S., Stevens, S., Wang, B., Sun, H., & Su, Y.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 28091-28114
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2306.06070
- OpenAlex
- W7133215859
- Cited by
- 134
- References
- 0
- Citations by year
- 2026: 104 · 2025: 14 · 2024: 15 · 2023: 1
- FWCI
- 34.44
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1942980, 2112606, CAREER)
- Topics
- Physical Sciences › Computer Science › Information Systems › Web Data Mining and Analysis
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Keywords
- web agents, web search, large language models, information retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
