OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent
Open accessAbstract
While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial retrieval. To bridge these gaps, we introduce OS-Symphony, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: (1) a Reflection-Memory Agent that utilizes milestone-driven long-term memory to enable trajectory-level self-correction, effectively mitigating visual context loss in long-horizon tasks; (2) Versatile Tool Agents featuring a Multimodal Searcher that adopts a SeeAct paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-Symphony delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online benchmarks, notably achieving 65.84% on OSWorld.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 3× · see citing works
“For visual state similarity S img , we employ a cascaded verification strategy: a Perceptual Hashing (pHash) (Zauner, 2010) check first filters distinct states using a tight Hamming distance threshold ( ≤ 1 ), followed by a Structural Similarity Index (SSIM) (Wang et al., 2004) calculation with a high threshold ( 0 .”
InfluentialSelf-citationInternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
Wang, W.-Y., Gao, Z., Gu, L.-X., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2508.18265 · Cited by 1,608
Self-citationSeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
Cheng, K., Sun, Q., Chu, Y., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2401.10935 · Cited by 690 · cited 1×
“…encompassing both general-purpose models (Ope-nAI, 2025a; Guo et al., 2025a; Anthropic, 2025b) adapted for agentic tasks and specialized models (Cheng et al., 2024; Wu et al., 2024b; Xu et al., 2024b; Qin et al., 2025; Wang et al., 2025c) fine-tuned on large-scale GUI datasets for dedicated…”
Self-citationReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608 · cited 1× · see citing works
“To further enhance the agent’s reasoning capabilities and support more complex decision-making processes (Yao et al., 2023), a "thought" component t i is incorporated prior to each action a i .”
OS-ATLAS: A Foundation Action Model for Generalist GUI AgentsPreprint
Wu, Z.-Y., Wu, Z.-Y., Xu, F.-Z., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2410.23218 · Cited by 405 · cited 1× · see citing works
“…both general-purpose models (Ope-nAI, 2025a; Guo et al., 2025a; Anthropic, 2025b) adapted for agentic tasks and specialized models (Cheng et al., 2024; Wu et al., 2024b; Xu et al., 2024b; Qin et al., 2025; Wang et al., 2025c) fine-tuned on large-scale GUI datasets for dedicated computer use.”
Self-citationOS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task SynthesisPreprint
Sun, Q., Cheng, K., Ding, Z., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2412.19723 · Cited by 174 · cited 1× · see citing works
“…of Vision-Language Models (VLMs) (Bai et al., 2025b,a; Wang et al., 2025b; Anthropic, 2025b; Hong et al., 2025), leading to vision-guided Computer-Using Agents (CUAs) (Sun et al., 2024b; Qin et al., 2025; Wang et al., 2025a; Xie et al., 2025a; Wu et al., 2025f; Liu et al., 2025b; Wu et al., 2025c).”
Self-citationA Survey on the Memory Mechanism of Large Language Model-based Agents
Zhang, Z., Dai, Q., Bo, X., Ma, C., Li, R., Chen, X., Zhu, J., Dong, Z., & Wen, J.-R. (2025) · ACM Transactions on Information Systems · doi:10.1145/3748302 · Cited by 118 · cited 1× · see citing works
“Efficient memory management is pivotal for long-horizon tasks (Hu et al., 2025b; Zhang et al., 2025d; Hu et al., 2025a; Wang et al., 2024; Sun et al., 2025a).”
GTA1: GUI Test-time Scaling Agent
Yang, Y., Li, D., Dai, Y.-T., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2507.05791 · Cited by 115
Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model CollaborationPreprint
Sun, Q., Yin, Z., Li, X., Wu, Z., Qiu, X.-P., & Kong, L.-P. (2023) · arXiv.org · doi:10.48550/arxiv.2310.00280 · Cited by 87 · cited 1× · see citing works
“Second, several works (Xu et al., 2024a; Sun et al., 2024c; Agashe et al., 2025a; Guo et al., 2025b; Xu et al., 2025) incorporate external knowledge via Retrieval-Augmented Generation (RAG) in an effort to generalize to unseen scenarios; however, as shown in Fig.”
Self-citationOSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Xie, T., Zhang, D., Chen, J., Li, X., Zhao, S., Cao, R., Hua, T. J., Cheng, Z., Shin, D., Lei, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.07972 · Cited by 64 · cited 4× · see citing works
“…responsible for task understanding and action prediction; (2) Tool Agents, consisting of Grounder , Coder , and Searcher , where the Searcher retrieves up-to-date tutorials in a human-like manner; and (3) The Reflection-Memory Agent , which compresses execution trajectories to maintain long-term…”
InfluentialOS Agents: A Survey on MLLM-based Agents for General Computing Devices Use
Hu, X., Xiong, T., Yi, B., Wei, Z., Xiao, R., Chen, Y.-R., Ye, J., Tao, M., Zhou, X., Zhao, Z., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2508.04482 · Cited by 59 · cited 3×
“While native CUAs (Zhang et al., 2024; Hu et al., 2025a; Liu et al., 2025b; Wang et al., 2025c) trained on large-scale computer-using trajectories are capable of digital navigation tasks, they often struggle to generalize to complex scenarios under a single-agent paradigm.”
Self-citationComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use AgentsPreprint
Lai, H., Liu, X., Zhao, Y., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2508.14040 · Cited by 49 · cited 1× · see citing works
“While some methods (Sun et al., 2024a; Song et al., 2025; Gonzalez-Pumariega et al., 2025) leverage general-purpose interfaces like code execution, others (Lai et al., 2025; Yang et al., 2025b; Jia et al., 2025b) depend on software-specific APIs via protocols such as MCP.”
API Agents vs. GUI Agents: Divergence and Convergence
Zhang, C.-Y., He, S., Li, L.-Q., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2503.11069 · Cited by 39
PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC
Liu, H.-W., Zhang, X., Xu, H.-Y., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2502.14282 · Cited by 39
The Unreasonable Effectiveness of Scaling Agents for Computer Use
Gonzalez-Pumariega, G., Tu, V., Lee, C.-L., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.02250 · Cited by 38
UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action
Yang, Y.-H., Yang, Z., Dou, Z.-Y., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.17790 · Cited by 34 · cited 1×
“While some methods (Sun et al., 2024a; Song et al., 2025; Gonzalez-Pumariega et al., 2025) leverage general-purpose interfaces like code execution, others (Lai et al., 2025; Yang et al., 2025b; Jia et al., 2025b) depend on software-specific APIs via protocols such as MCP.”
BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanism
Wu, Q., Gao, P.-Z., Liu, W., et al. (2025) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2505.20660 · Cited by 29 · cited 1×
“In parallel, CUA frameworks (Wu et al., 2024a; Agashe et al., 2025b; Yang et al., 2025a; Wu et al., 2025d; Ye et al., 2025a; Zhang et al., 2025c) prioritize modularity by decomposing complex tasks.”
Jia, C.-Y., Luo, M., Dang, Z., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2410.18603 · Cited by 29 · cited 1×
“…2025; Gonzalez-Pumariega et al., 2025; Guo et al., 2025b) have emerged by orchestrating multiple specialized sub-agents, e.g. , planner, grounder, and coder (Chen et al., 2025; Jia et al., 2025a), to coordinate seamlessly, exhibiting significant potential for developing reliable generalist CUAs.”
Self-citationWorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point
Zhao, H. H., Yang, K., Yu, W., et al. (2025) · doi:10.48550/arxiv.2502.08047 · Cited by 25 · cited 1×
“In parallel, other efforts construct task-specific knowledge databases or training corpora by manually curating software documentation (Xu et al., 2024a, 2025) and video tuto-rials (Zhao et al., 2025).”
DeepAgent: A General Reasoning Agent with Scalable Toolsets
Li, X., Jiao, W., Jin, J., Dong, G., Jin, J., Wang, Y., Wang, H., Zhu, Y., Wen, J.-R., Yuan, L., et al. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792460 · Cited by 22 · cited 1× · see citing works
“Other approaches leverage multi-scale memory (Wu et al., 2024a; Li et al., 2025b), fusing short-term working memory and long-term procedural memory to maintain a comprehensive repository of trajectories and acquired knowledge.”
Self-citationGrounding Computer Use Agents on Human Demonstrations
Feizi, A., Nayak, S., Jian, X.-R., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2511.07332 · Cited by 22 · cited 1×
“5-72B , Holo2-30B-A3B (Company, 2025), GTA1-32B (Yang et al., 2025a) and GroundNext-7B (Feizi et al., 2025).”
Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents
Wang, Z., Li, X.-J., Ye, Y., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.23691 · Cited by 22
Self-citationAi, W.-W. (2012) · doi:10.1515/9781400845859-005 · Cited by 21
R-WoM: Retrieval-augmented World Model For Computer-use Agents
Mei, K., Guo, J., Chang, S.-C., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.11892 · Cited by 16 · cited 1×
“Early lines of research (Shi et al., 2025; Agashe et al., 2025a; Mei et al., 2025; Guo et al., 2025b) rely on general-purpose AI search engines ( e.g. , Perplexica 1 ) to perform static knowledge retrieval prior to task execution.”
Retrieval-augmented GUI Agents with Generative Guidelines
Xu, R., Ma, K.-X., Yu, W.-H., et al. (2025) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2509.24183 · Cited by 16 · cited 1×
“Second, several works (Xu et al., 2024a; Sun et al., 2024c; Agashe et al., 2025a; Guo et al., 2025b; Xu et al., 2025) incorporate external knowledge via Retrieval-Augmented Generation (RAG) in an effort to generalize to unseen scenarios; however, as shown in Fig.”
Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents
Agashe, S., Wong, K., Tu, V., Yang, J., Li, A., & Wang, X. E. (2025) · arXiv · doi:10.48550/arxiv.2504.00906 · Cited by 14 · cited 1× · see citing works
“In parallel, CUA frameworks (Wu et al., 2024a; Agashe et al., 2025b; Yang et al., 2025a; Wu et al., 2025d; Ye et al., 2025a; Zhang et al., 2025c) prioritize modularity by decomposing complex tasks.”
AgentProg: Empowering Long-Horizon GUI Agents with Program-guided Context Management
Tian, S.-Z., Wen, H., Chen, Y., et al. (2025) · ACM SIGMOBILE International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3745756.3809245 · Cited by 14 · cited 2×
“First, while memory modules (Song et al., 2025; Cheng et al., 2025; Tian et al., 2025) are employed to support long-horizon tasks, current context management mechanisms often lack granular control over historical visual information curation and pruning.”
MGA: Memory-Driven GUI Agent for Observation-Centric InteractionPreprint
Cheng, W., Ni, E., Wang, W.-L., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.24168 · Cited by 13 · cited 2× · see citing works
“First, while memory modules (Song et al., 2025; Cheng et al., 2025; Tian et al., 2025) are employed to support long-horizon tasks, current context management mechanisms often lack granular control over historical visual information curation and pruning.”
Breaking the Data Barrier - Building GUI Agents Through Task GeneralizationPreprint
Zhang, J.-L., Ding, Z., Ma, C., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2504.10127 · Cited by 12 · cited 1× · see citing works
“In parallel, CUA frameworks (Wu et al., 2024a; Agashe et al., 2025b; Yang et al., 2025a; Wu et al., 2025d; Ye et al., 2025a; Zhang et al., 2025c) prioritize modularity by decomposing complex tasks.”
Self-citationGPT-4V(ision) is a Generalist Web Agent, if Grounded
Zheng, B., Gou, B., Kil, J., Sun, H., & Su, Y. (2024) · arXiv · doi:10.48550/arxiv.2401.01614 · Cited by 10 · cited 1× · see citing works
“So we introduce a Visual-Centric Search as a Tool paradigm, where the Searcher adopts a VLM-driven See-Act (Zheng et al., 2024) strategy to interact with rendered pages and synthesize tutorials.”
Agentic Lybic: Multi-Agent Execution System with Tiered Reasoning and Orchestration
Guo, L.-X., Zhu, B., Tao, Q., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2509.11067 · Cited by 10 · cited 2×
“Consequently, modular CUA frameworks (Wu et al., 2024a; Song et al., 2025; Gonzalez-Pumariega et al., 2025; Guo et al., 2025b) have emerged by orchestrating multiple specialized sub-agents, e.g. , planner, grounder, and coder (Chen et al., 2025; Jia et al., 2025a), to coordinate seamlessly,…”
Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis
Xie, T., Deng, J., Li, X., Yang, J., Wu, H., Chen, J., Hu, W., Wang, X., Xu, Y., Wang, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.13227 · Cited by 7 · cited 4×
“5-72B , Holo2-30B-A3B (Company, 2025), GTA1-32B (Yang et al., 2025a) and GroundNext-7B (Feizi et al., 2025).”
InfluentialSelf-citationNested Browser-Use Learning for Agentic Information Seeking
Li, B., Wu, J.-L., Yin, W., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2512.23647 · Cited by 6 · cited 1×
“However, purely text-based retrieval is insufficient for GUI scenarios, as it struggles to interpret verbose HTML and screenshot-heavy tutorials (Li et al., 2025a).”
OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic ModelsPreprint
Wu, Z.-Y., Xie, J., Li, Z.-H., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2512.16295 · Cited by 6 · cited 1× · see citing works
“…of Vision-Language Models (VLMs) (Bai et al., 2025b,a; Wang et al., 2025b; Anthropic, 2025b; Hong et al., 2025), leading to vision-guided Computer-Using Agents (CUAs) (Sun et al., 2024b; Qin et al., 2025; Wang et al., 2025a; Xie et al., 2025a; Wu et al., 2025f; Liu et al., 2025b; Wu et al., 2025c).”
Self-citationCristescu, H., Park, C., Nguyen, T., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2511.17131 · Cited by 6 · cited 1×
“To strengthen the generalization of CUAs, an increasing number of studies (Zhang et al., 2024; Nguyen et al., 2025; Hu et al., 2025a) has integrated Retrieval-Augmented Generation (RAG) to access external knowledge.”
Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale
Bonatti, R., Zhao, D., Bonacci, F., Dupont, D., Abdali, S., Li, Y., Lu, Y., Wagle, J., Koishida, K., Bucker, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.08264 · Cited by 3 · cited 1× · see citing works
“Additionally, the EasyOCR width threshold is configured to 0.1 to ensure precise word-level recognition.”
InfluentialAgent S: An Open Agentic Framework that Uses Computers Like a Human
Agashe, S., Han, J., Gan, S., Yang, J., Li, A., & Wang, X. E. (2024) · arXiv · doi:10.48550/arxiv.2410.08164 · Cited by 2 · see citing works
Scaling Long-Horizon LLM Agent via Context-Folding
Sun, W., Lu, M., Ling, Z., Liu, K., Yao, X., Yang, Y., & Chen, J. (2025) · arXiv · doi:10.48550/arxiv.2510.11967 · Cited by 1 · cited 1× · see citing works
“Recent LLM-based methods leverage history summarization (Wu et al., 2025e; Yu et al., 2025) or context folding (Ye et al., 2025b; Sun et al., 2025b) to compress interactions.”
ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data
Liu, Z., Xie, J., Ding, Z., Li, Z., Yang, B., Wu, Z., Wang, X., Sun, Q., Liu, S., Wang, W., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.15221
Self-citationAguvis: Unified Pure Vision Agents for Autonomous GUI InteractionPreprint
Xu, Y., Wang, Z., Wang, J., Lu, D., Xie, T., Saha, A., Sahoo, D., Yu, T., & Xiong, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.04454
Self-citationUI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement LearningPreprint
Wang, H., Zou, H., Song, H., Feng, J., Fang, J., Lu, J., Liu, L., Luo, Q., Liang, S., Huang, S., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.02544 · cited 3×
“Beyond architectural paradigms, the field is transitioning from purely vision-based approaches (Zhang et al., 2025a,b; Wang et al., 2025a) toward hybrid GUI-API strategies.”
Self-citationAgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials
Xu, Y., Lu, D., Shen, Z., Wang, J., Wang, Z., Mao, Y., Xiong, C., & Yu, T. (2024) · arXiv · doi:10.48550/arxiv.2412.09605
Self-citationGUI Knowledge Bench: Revealing the Knowledge Gap of VLMs in GUI TasksPreprint
Shi, C., Yu, Z., Gao, Z., Feng, R., Liu, E., Wu, Y., Jia, Y., Xiang, L., He, Z., & Li, Q. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.26098 · cited 1×
“Early lines of research (Shi et al., 2025; Agashe et al., 2025a; Mei et al., 2025; Guo et al., 2025b) rely on general-purpose AI search engines ( e.g. , Perplexica 1 ) to perform static knowledge retrieval prior to task execution.”
Self-citation
Flagged first, then by citations
15 of the 58 references couldn’t be matched to a known paper.
Research works printed but not linked: 31
- Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, and 1 others. 2025a. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. · arXiv
- Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wen- bin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025b. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. · arXiv
- Xuetian Chen, Yinghao Chen, Xinfeng Yuan, Zhuo Peng, Lu Chen, Yuekeng Li, Zhoujia Zhang, Yingqian Huang, Leyan Huang, Jiaqing Liang, and 1 others. 2025. Os-map: How far can computer- using agents go in breadth and depth? arXiv preprint arXiv:2507.19132. · arXiv
- Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Mar- cel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. · arXiv
- Tianyu Fu, Anyang Su, Chenxu Zhao, Hanning Wang, Minghui Wu, Zhe Yu, Fei Hu, Mingjia Shi, Wei Dong, Jiayao Wang, and 1 others. 2025. Mano technical report. arXiv preprint arXiv:2509.17336. · arXiv
- Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, and 1 others. 2025a. Seed1. 5-vl technical report. arXiv preprint arXiv:2505.07062. · arXiv
- v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006. · arXiv
- Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, Jiejun Tan, Yan- bin Yin, Jiongnan Liu, Zeyu Zhang, Zhongxiang Sun, Yutao Zhu, Hao Sun, Boci Peng, and 28 others. 2025b. Memory in the age of ai agents. Preprint, arXiv:2512.13564. · arXiv
- Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. · arXiv
- Hongrui Jia, Jitong Liao, Xi Zhang, Haiyang Xu, Tian- bao Xie, Chaoya Jiang, Ming Yan, Si Liu, Wei Ye, and Fei Huang. 2025b. Osworld-mcp: Bench- marking mcp tool invocation in computer-use agents. arXiv preprint arXiv:2510.24563. · arXiv
- Dang Nguyen, Jian Chen, Yu Wang, Gang Wu, Namy- ong Park, Zhengmian Hu, Hanjia Lyu, Junda Wu, Ryan Aponte, Yu Xia, and 1 others. 2025. Gui agents: A survey. In Findings of the Association for Compu- tational Linguistics: ACL 2025, pages 22522-22538.
- Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, and 1 others. 2025. Ui- tars: Pioneering automated gui interaction with native agents. arXiv preprint arXiv:2501.12326. · arXiv
- Linxin Song, Yutong Dai, Viraj Prabhu, Jieyu Zhang, Taiwei Shi, Li Li, Junnan Li, Silvio Savarese, Zeyuan Chen, Jieyu Zhao, and 1 others. 2025. Coact-1: Computer-using agents with coding as actions. arXiv preprint arXiv:2508.03923. · arXiv
- Qiushi Sun, Zhirui Chen, Fangzhi Xu, Kanzhi Cheng, Chang Ma, Zhangyue Yin, Jianing Wang, Chengcheng Han, Renyu Zhu, Shuai Yuan, and 1 others. 2024a. A survey of neural code intelligence: Paradigms, advances and beyond. arXiv preprint arXiv:2403.14734. · arXiv
- Qiushi Sun, Zhoumianze Liu, Chang Ma, Zichen Ding, Fangzhi Xu, Zhangyue Yin, Haiteng Zhao, Zhenyu Wu, Kanzhi Cheng, Zhaoyang Liu, and 1 others. 2025a. Scienceboard: Evaluating multimodal au- tonomous agents in realistic scientific workflows. arXiv preprint arXiv:2505.19897. · arXiv
- Zeyi Sun, Yuhang Cao, Jianze Liang, Qiushi Sun, Ziyu Liu, Zhixiong Zhang, Yuhang Zang, Xiaoyi Dong, Kai Chen, Dahua Lin, and 1 others. 2025c. Coda: Coordinating the cerebrum and cerebellum for a dual- brain computer use agent with decoupled reinforce- ment learning. arXiv preprint arXiv:2508.20096. · arXiv
- Zhengwei Tao, Jialong Wu, Wenbiao Yin, Junkai Zhang, Baixuan Li, Haiyang Shen, Kuan Li, Li- wen Zhang, Xinyu Wang, Yong Jiang, and 1 others. 2025. Webshaper: Agentically data synthesizing via information-seeking formalization. arXiv preprint arXiv:2507.15061. · arXiv
- Tongyi DeepResearch Team, Baixuan Li, Bo Zhang, Dingchu Zhang, Fei Huang, Guangyu Li, Guoxin Chen, Huifeng Yin, Jialong Wu, Jingren Zhou, and 1 others. 2025. Tongyi deepresearch technical report. arXiv preprint arXiv:2510.24701. · arXiv
- Shuai Wang, Weiwen Liu, Jingxuan Chen, Yuqi Zhou, Weinan Gan, Xingshan Zeng, Yuhan Che, Shuai Yu, Xinlong Hao, Kun Shao, and 1 others. 2024. Gui agents with foundation models: A comprehensive survey. arXiv preprint arXiv:2411.04890. · arXiv
- Xinyuan Wang, Bowen Wang, Dunjie Lu, Junlin Yang, Tianbao Xie, Junli Wang, Jiaqi Deng, Xiaole Guo, Yiheng Xu, Chen Henry Wu, Zhennan Shen, Zhuokai Li, Ryan Li, Xiaochuan Li, Junda Chen, Boyuan Zheng, Peihang Li, Fangyu Lei, Ruisheng Cao, and 23 others. 2025c. Opencua: Open foundations for computer-use agents. Preprint, arXiv:2508.09123. · arXiv
- Xuehui Wang, Zhenyu Wu, JingJing Xie, Zichen Ding, Bowen Yang, Zehao Li, Zhaoyang Liu, Qingyun Li, Xuan Dong, Zhe Chen, and 1 others. 2025d. Mmbench-gui: Hierarchical multi-platform evalu- ation framework for gui agents. arXiv preprint arXiv:2507.19478. · arXiv
- Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Gang Fu, Yong Jiang, and 1 others. 2025a. Web- dancer: Towards autonomous information seeking agency. arXiv preprint arXiv:2505.22648. · arXiv
- Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, and 1 others. 2025b. Web- walker: Benchmarking llms in web traversal. arXiv preprint arXiv:2501.07572. · arXiv
- Qianhui Wu, Kanzhi Cheng, Rui Yang, Chaoyun Zhang, Jianwei Yang, Huiqiang Jiang, Jian Mu, Baolin Peng, Bo Qiao, Reuben Tan, and 1 others. 2025c. Gui-actor: Coordinate-free visual grounding for gui agents. arXiv preprint arXiv:2506.03143. · arXiv
- Xixi Wu, Kuan Li, Yida Zhao, Liwen Zhang, Litu Ou, Huifeng Yin, Zhongwang Zhang, Xinmiao Yu, Dingchu Zhang, Yong Jiang, and 1 others. 2025e. Resum: Unlocking long-horizon search in- telligence via context summarization. arXiv preprint arXiv:2509.13313. · arXiv
- Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. 2024a. Os-copilot: Towards gener- alist computer agents with self-improvement. arXiv preprint arXiv:2402.07456. · arXiv
- Jiabo Ye, Xi Zhang, Haiyang Xu, Haowei Liu, Jun- yang Wang, Zhaoqing Zhu, Ziwei Zheng, Feiyu Gao, Junjie Cao, Zhengxi Lu, and 1 others. 2025a. Mobile- agent-v3: Foundamental agents for gui automation. arXiv preprint arXiv:2508.15144. · arXiv
- Rui Ye, Zhongwang Zhang, Kuan Li, Huifeng Yin, Zhengwei Tao, Yida Zhao, Liangcai Su, Liwen Zhang, Zile Qiao, Xinyu Wang, and 1 others. 2025b. Agentfold: Long-horizon web agents with proactive context management. arXiv preprint arXiv:2510.24699. · arXiv
- Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei- Ying Ma, Jingjing Liu, Mingxuan Wang, and 1 others. 2025. Memagent: Reshaping long-context llm with multi-conv rl-based memory agent. arXiv preprint arXiv:2507.02259. · arXiv
- Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Guyue Liu, Qingwei Lin, and 1 others. 2024. Large lan- guage model-brained gui agents: A survey. arXiv preprint arXiv:2411.18279. · arXiv
- Chaoyun Zhang, He Huang, Chiming Ni, Jian Mu, Si Qin, Shilin He, Lu Wang, Fangkai Yang, Pu Zhao, Chao Du, and 1 others. 2025b. Ufo2: The desktop agentos. arXiv preprint arXiv:2504.14603. · arXiv
Cited by
Sorted by citations
Natural-Language Agent Harnesses
Pan, L., Zou, L., Guo, S., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2603.25723 · Cited by 58
MGA: Memory-Driven GUI Agent for Observation-Centric InteractionPreprint
Cheng, W., Ni, E., Wang, W.-L., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.24168 · Cited by 13 · see citing works
“For example, OS-Symphony and Agent S3 [13, 39] integrate multiple MLLMs with orchestrators, reflection modules, and tool chains.”
OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis
Cheng, K., Li, Z.-H., Ma, Z., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2604.15093 · Cited by 12
Self-citationAgent Alpha: Tree Search Unifying Generation, Exploration and Evaluation for Computer-Use Agents
Tang, S.-Z., Chen, R.-Q., & Lan, T. (2026) · arXiv.org · doi:10.48550/arxiv.2602.02995 · Cited by 12
“We compare the performance and efficiency of Agent Alpha against state-of-the-art methods listed on the OSWorld leaderboard, including UiPath Screen Agent, Claude Sonnet 4.5, Agent S3 [3], CoAct-1 [33], GTA1 [34], OS-symphony [35]) and GBOX Agent.”
InfluentialToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agents
Hu, X., Zhang, X., Xu, H.-Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2605.12481 · Cited by 11
“Current CUAs generally follow two paradigms: multi-agent systems [1, 9, 16, 32, 51, 53], which decompose tasks across specialized modules, and end-to-end agentic models [18, 28, 40, 46, 48], which integrate planning and grounding within a unified policy.”
OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task ExecutionPreprint
Zhang, L., Xiao, Y., Lu, X., Cao, J., Zhao, Y., Zhou, J., An, L., Feng, Z., Sha, W., Shi, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.20380
“OS-Symphony [31] and GTA1 [32] both advance computer-using agent frameworks by improving robustness and generalization for GUI-based tasks through careful system design and enhanced inference-time scaling.”
StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure
Wu, W.-Y., Zhu, S.-B., Zhou, K., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.11388 · Cited by 8
“Consequently, the agent struggles to maintain a clear causal understanding of task progress, making it difficult to promptly detect abnormal states, identify intermediate errors, and perform accurate planning and execution [8, 9].”
VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation
Han, Q.-J., Tu, H.-Q., Wang, Z.-J., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2604.21375 · Cited by 7
“OS-Symphony (Yang et al., 2026a) and GTA1 (Yang et al., 2026b) combine memory systems and test-time search, while CoAct-1 (Song et al., 2026) and EvoCUA (Xue et al., 2026) emphasize coding-as-actions and synthetic experience.”
InfluentialSelf-citationMacAgentBench: Benchmarking AI Agents on Real-World macOS Desktop
Fu, Y., Fu, B.-W., Wu, Z.-Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.22557 · Cited by 4
“…control capabilities (Anthropic, 2024, 2025; Wang et al., 2025a; Bai et al., 2025; GLM-5-Team et al., 2026; Qin et al., 2025; Hong et al., 2024), and a growing ecosystem of CUA frameworks and models has formed (Yao et al., 2023; Yang et al., 2026; Agashe et al., 2025a,b; Openclaw Team, 2025).”
Self-citationXie, R., Gao, Z., Shi, C., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2603.26266 · Cited by 4
“Recent works extend this paradigm to GUI agents: RAG-GUI [28] retrieves web tutorials, KG-RAG [11] constructs knowledge graphs from UI transitions for plan retrieval, and OS-Symphony [29] synthesizes visually grounded tutorials on demand.”
Self-citationMAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation
Yan, H., Gu, Z., Zhou, B.-T., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.29320 · Cited by 3
OpenMobile-2: Building Versatile Mobile Agents with Scalable Environments and App-Native Tools
Yan, C.-Y., Zhang, Y.-Y., Cheng, K., et al. (2026) · doi:10.48550/arxiv.2610.14157 · Cited by 1
Self-citationSynChain: Inducing Computer-Use Agent Systems to Construct Their Own Attack Chains
Zhang, F.-Y., Zhang, J.-M., Wang, C., et al. (2026) · doi:10.48550/arxiv.2608.06862 · Cited by 1
A History-Aware Visually Grounded Critic for Computer Use Agents
Lee, J., Khan, Z., Prasad, A., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.11078 · Cited by 1
Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents?
Zhao, Y., Ding, Z., Wu, J.-Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2605.27881 · Cited by 1
Self-citationOS-Themis: A Scalable Critic Framework for Generalist GUI Rewards
Li, Z., Wu, Z., Zhao, Y., Yang, B., Xie, J., Liu, Z., Liu, Z., Jin, K., Liang, J., Li, Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2603.19191
Self-citationOS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents
Yang, B., Jin, K., Ding, Z., et al. (2026) · doi:10.48550/arxiv.2610.22234
Self-citationGUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolution
Yang, G.-Y., Qu, Z.-K., Li, X., et al. (2026) · doi:10.48550/arxiv.2610.00948
“…experience retrieval, specialized grounding, and coding actions (Agashe et al., 2025a;b; Gonzalez-Pumariega et al., 2026b); CoAct-1 routes between GUI and programmatic execution, and OS-Symphony co-ordinates planning, grounding, search, coding, and memory (Song et al., 2026; Yang et al., 2026a).”
Self-citationAnyAct: Universal Action for Self-Evolving Agents
Xu, L.-R., Jiang, Y., Zhang, J.-C., et al. (2026) · doi:10.48550/arxiv.2609.37025
“Current research typically depends either on a model’s inherent GUI proficiency (Lu et al., 2024; Anthropic, 2025a; Wang et al., 2025a) or on agentic frameworks (Yang et al., 2026; Song et al., 2025; Xie et al., 2025).”
Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization
Bai, B., Yuan, J., Wu, H.-Z., et al. (2026) · doi:10.48550/arxiv.2609.02309
“OS-Symphony and LongHori-zonUI extend this systems view to long-horizon robustness, combining reflection, memory, tutorial retrieval, element-indexed perception, or rollback-based execution to keep trajectories recoverable in unseen or sustained tasks (Yang et al., 2026a; Kang et al., 2026).”
LocalLSTC: A Long Short-Term Control Architecture for Locally Deployed GUI Agents
Li, W., Paik, H., & Sui, Y.-L. (2026) · doi:10.48550/arxiv.2608.25777
“5-7B ( Qin et al., 2025) 27.4 27.3 24.5 - 15.9 - OpenCUA-32B (Wang et al., 2025c) 34.8 34.1 29.7 - - - DeepMiner-Mano-7B (Fu et al., 2025) 40.2 - - - - - OpenCUA-72B-Preview (Wang et al., 2025c) 45.0 44.9 39.0 - - - OS-Symphony / Qwen3-VL-32B (Yang et (Fu et al., 2025) 53.9 - - - - - GUI-Owl-1.”
InfluentialGlobal Commander and Local Operative: A Dual-Agent Framework for Scene Navigation
Jin, K., Wu, Y., Wu, S.-Q., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.18941
“Consequently, recent research explores multi-agent collaboration, leveraging role-specialized agents and structured communication to partition cognitive workloads and enhance reliability [60–62].”
Self-citation
- Authors
- Yang, B., Jin, K., Wu, Z., Liu, Z., Sun, Q., Li, Z., Xie, J., Liu, Z., Xu, F., Cheng, K., Li, Q., Wang, Y., Qiao, Y., Wang, Z., & Ding, Z.
- Affiliations
- University of Science and Technology of China (CN)
- National University of Singapore (SG)
- Shanghai Artificial Intelligence Laboratory (CN)
- Hong Kong University of Science and Technology (HK)
- University of Hong Kong (HK)
- Chinese University of Hong Kong (HK)
- Xi'an Jiaotong University (CN)
- Nanjing University (CN)
- Harbin Institute of Technology (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2601.07779
- OpenAlex
- W7123960300
- Semantic Scholar
- a73541ae03633322a0bb5f75ec9d34daf432573f
- Cited by
- 0
- References
- 58
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- vision-language models, self-correction, multimodal retrieval, agent robustness, generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
