EvoSkill: Automated Skill Discovery for Multi-Agent Systems
Open accessAbstract
Coding agents are increasingly used as general-purpose problem solvers, but their flexibility does not by itself confer the domain expertise needed for specialized tasks. Recent work addresses this through \textit{agent skills}: reusable workflows, and code, that augment agents with domain-specific capabilities. Most skills today are hand-crafted, and existing evolutionary approaches optimize low-level artifacts (e.g. prompts \& code) that are tightly coupled to specific models and tasks. We introduce \textbf{EvoSkill}, a self-evolving framework that automatically discovers and refines agent skills through iterative failure analysis. EvoSkill analyzes execution failures, proposes new skills or edits to existing ones, and materializes them into structured, reusable skill folders. A Pareto frontier of agent programs governs selection, retaining only skills that improve held-out validation performance while the underlying model remains frozen. We evaluate EvoSkill on two benchmarks: OfficeQA, a grounded reasoning benchmark over U.S.\ Treasury data, where it improves exact-match accuracy by \textbf{7.3\%} (60.6\% $\to$ 67.9\%); and SealQA, a search-augmented QA benchmark with noisy retrieval, where it yields a \textbf{12.1\%} gain (26.6\% $\to$ 38.7\%). We also investigate the zero-shot transfer capabilties of skills evolved on one task to the other; in particular: skills evolved from SealQA transfers zero-shot to BrowseComp, improving accuracy by \textbf{5.3\%} without modification demonstrating that skill-level optimization produces transferable capabilities beyond the training task.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior skill-evolution methods do not maintain what has been learned as a separate, evolving knowledge representation.
“However, these methods do not maintain what has been learned as a separate, evolving knowledge representation.”
From WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution (2026) · cites this paper as Alzubi et al., 2026
The Introduction (page unavailable) says earlier evolutionary approaches optimize prompts or code and do not naturally yield reusable components that transfer across settings. The Methodology contrasts this with EvoSkill’s evolving skill repository. The source supports the distinction, but it describes those earlier approaches as artifact-evolution methods, not prior skill-evolution methods, and does not characterize their output as a separate knowledge representation. (LLM, conf=0.84)
Experimental — accuracy not yet measuredBenchmark validation splits are relatively small, which can introduce evaluation noise into gating decisions.
“Following established setups in prior work (Alzubi et al., 2026; Yang et al., 2026), benchmark validation splits are relatively small, which can introduce evaluation noise into gating decision.”
From WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution (2026) · cites this paper as Alzubi et al., 2026
Section 3.1.2 (page unavailable) reports a validation set of 17 examples (about 7%) used for frontier selection, and the Environment Branches section says validation scores determine whether a child enters the frontier. The paper does not state that the small split introduces evaluation noise into those decisions; that is an inference rather than a finding it reports. (LLM, conf=0.84)
Experimental — accuracy not yet measuredIn prior self-improvement systems such as EvoSkill, GEPA, and SkillWeaver, the meta-level logic is hardcoded in advance and shared by every branch throughout the run.
“These systems, however, evolve only what the agent does, not how it evolves : the artifact under optimization changes while the operator that optimizes it stays fixed. In the vocabulary of selfimproving machines (Good, 1965; Schmidhuber, 2006), they are self-improving but stop short of being recursively self-improving. The meta-level logic is hardcoded in advance and shared by every branch through”
From MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution (2026) · cites this paper as Alzubi et al., 2026
Section 2.3.2 (Environment Setup; page unavailable) says EvoSkill’s codebase is fixed and its branches differ only in skills and metadata, supporting the claim for EvoSkill. The paper discusses GEPA’s artifact optimization but does not establish that GEPA shares fixed meta-level logic across branches; the supplied passages do not address SkillWeaver. (LLM, conf=0.83)
Experimental — accuracy not yet measuredExisting skill-evolution methods do not maintain what has been learned as a separate, evolving knowledge representation.
“However, these methods do not maintain what has been learned as a separate, evolving knowledge representation.”
From WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution (2026) · cites this paper as Alzubi et al., 2026
The Introduction and §4.2 say that earlier evolutionary approaches optimize prompts or code rather than persistent, reusable skills; §2 says EvoSkill instead evolves a skill repository while keeping the model frozen. These passages support the contrast, but they do not establish the broader claim that existing skill-evolution methods lack any separate, evolving knowledge representation. Our copy provides no page numbers for these passages. (LLM, conf=0.78)
Experimental — accuracy not yet measuredAll task splits and available toolsets are strictly matched with prior work.
“All task splits and available toolsets are strictly matched with prior work (Alzubi et al., 2026; Yang et al., 2026).”
From WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution (2026) · cites this paper as Alzubi et al., 2026
Sections 2.3.1, 3.1.2, and 3.2.2 describe EvoSkill’s task partitions, including configurable splits and use of SealQA’s seal-0 split. They do not say that all task splits or available toolsets were strictly matched with prior work. Searches for matched splits, toolsets, and baseline tool access found no passage establishing that comparison. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models
Pham, T., Nguyen, N., Zunjare, P., Chen, W., Tseng, Y.-M., & Vu, T. (2025) · arXiv · doi:10.48550/arxiv.2506.01062 · cited 1×
“SealQA [10] is a challenge benchmark for evaluating search-augmented language models on fact-seeking questions where web search yields conflicting, noisy, or unhelpful results.”
Self-citation
Flagged first, then by citations
Research works printed but not linked: 9
- Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl- Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. Gepa: Reflective prompt evolution can outperform reinforcement learning, 2026. URL https: //arxiv.org/abs/2507.19457.
- Salaheddin Alzu'bi, Baran Nama, Arda Kaz, Anushri Eswaran, Weiyuan Chen, Sarvesh Khetan, Rishab Bala, Tu Vu, and Sewoong Oh. Roma: Recursive open meta-agent framework for long-horizon multi-agent systems, 2026. URL https://arxiv.org/abs/2602.01848. · arXiv
- Yoonho Lee, Joseph Boen, and Chelsea Finn. Feedback descent: Open-ended text optimization via pairwise comparison, 2025. URL https://arxiv.org/abs/2511.07919. · arXiv
- Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback, 2023. URL https://arxiv.org/abs/2303.17651. · arXiv
- Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. Alphaevolve: A coding agent for scientific and algorithmic discovery, 2025. URL https://arxiv.org/abs/2506.13131. · arXiv
- Tu Vu, Kalpesh Krishna, Salaheddin Alzubi, Chris Tar, Manaal Faruqui, and Yun-Hsuan Sung. Foundational autoraters: Taming large language models for better automatic evaluation, 2024. URL https://arxiv.org/abs/2407.10817. · arXiv
- Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 2023. URL https://arxiv.org/abs/2305.16291. · arXiv
- Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. Openhands: An open platform for ai software developers as generalist agents, 2025. URL https://arxiv.org/abs/2407.16741. · arXiv
- Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents, 2025. URL https://arxiv.org/abs/2504. 12516.
Cited by
Sorted by citations
DarwinX: Evolving Agent Harnesses Through Natural Selection
Zhang, Y., Dai, Y., Tan, J., Yang, L., Mullur, R., Hoang, T., Hu, Z., Zhu, J., Mui, P., Savarese, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2608.07545 · Cited by 1 · see citing works
“Workflow systems lift the artifact to a graph of otherwise-fixed components [15, 58, 61], and skill-centric agents accumulate a reusable external library [3, 45, 51].”
SkillOpt: Executive Strategy for Self-Evolving Agent Skills
Yang, Y., Gong, Z., Huang, W., Yang, Q., Zhou, Z., Huang, Z., Li, Y., Gao, X., Dai, Q., Liu, B., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.23904 · Cited by 2 · see citing works
Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents
Zhang, X., Wang, G., Cui, Y., Qiu, W., Li, Z., Zhu, B., & He, P. (2026) · arXiv
“We (1) formalize the Experience Compression Spectrum, unifying agent memory, skills, and rules on a single compression axis (Section 2); (2) map 20+ systems onto this spectrum, revealing that none supports adaptive cross-level compression—the “missing diagonal” (Section 2); (3) quantify the…”
WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution
Tang, L., Rashtchian, C., Ferng, C.-S., Tomkins, A., Juan, D.-C., & Vu, T. (2026) · arXiv · Cited by 1 · see citing works
“We adopt this strict criterion following prior skill-evolution frameworks (Alzubi et al., 2026; Yang et al., 2026) to ensure a fair comparison.”
InfluentialSelf-citationMetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution
Wang, Z., Yan, M., Bi, J., Yan, S., Tresp, V., & Ma, Y. (2026) · arXiv · doi:10.48550/arxiv.2607.05297
VERSE: Verified Self-Evolving Optimizer for Agent Harnesses
Wang, Z., Ge, Y., Wang, Z., Wang, H., Xu, Y., Frandsen, J., Fu, S., Wilson, A. C., & Reddy, C. K. (2026) · arXiv · doi:10.48550/arxiv.2610.02616
Managing Procedural Memory in LLM Agents: Control, Adaptation, and Evaluation
Belikova, J., Parchiev, R., Egorov, E., Davydenko, G., Gusev, G., Savchenko, A., & Makarenko, M. (2026) · arXiv · doi:10.48550/arxiv.2606.23127
- Authors
- Alzubi, S., Provenzano, N., Bingham, J., Chen, W., & Vu, T.
- Affiliations
- Virginia Tech (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2603.02766
- OpenAlex
- W7133528385
- Cited by
- 1
- References
- 1
- Citations by year
- 2026: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- agent skills, multi-agent systems, evolutionary algorithms, failure analysis, Pareto front, skill transfer, zero-shot transfer, coding agents, grounded reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
