The DOI matches its DataCite record; 85 of the 99 references it cites were found in a registry or an index; 2 of the 6 sources its claims cite were checked for support.
Multi-agent Architecture Search via Agentic Supernet
Open accessAbstract
Large Language Model (LLM)-empowered multi-agent systems extend the cognitive boundaries of individual agents through disciplined collaboration and interaction, while constructing these systems often requires labor-intensive manual designs. Despite the availability of methods to automate the design of agentic workflows, they typically seek to identify a static, complex, one-size-fits-all system, which, however, fails to dynamically allocate inference resources based on the difficulty and domain of each query. To address this challenge, we shift away from the pursuit of a monolithic agentic system, instead optimizing the \textbf{agentic supernet}, a probabilistic and continuous distribution of agentic architectures. We introduce MaAS, an automated framework that samples query-dependent agentic systems from the supernet, delivering high-quality solutions and tailored resource allocation (\textit{e.g.}, LLM calls, tool calls, token cost). Comprehensive evaluation across six benchmarks demonstrates that MaAS \textbf{(I)} requires only $6\sim45\%$ of the inference costs of existing handcrafted or automated multi-agent systems, \textbf{(II)} surpasses them by $0.54\%\sim11.82\%$, and \textbf{(III)} enjoys superior cross-dataset and cross-LLM-backbone transferability.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
MaAS reframes automated multi-agent design as optimizing an “agentic supernet,” a probabilistic distribution over agentic architectures, and samples query-dependent systems at inference. Across six benchmarks it improves accuracy over handcrafted and automated baselines while using substantially fewer tokens and lower cost.
Key findings
- MaAS achieves the best average on GSM8K, MATH, MultiArith, HumanEval, and MBPP: 83.59 average, with gains over baselines of 0.54%–16.89%; e.g., MATH 51.82 vs. AFlow 51.28, HumanEval 92.85 vs. AFlow 90.93, MBPP 82.17 vs. AFlow 81.67.
- On GAIA, MaAS reaches 20.69 average vs. AFlow 8.00, AgentSquare 16.34, and GPTSwarm 16.33; Level 1 25.91, Level 2 22.01, Level 3 6.25.
- On MATH, MaAS uses $3.38 training cost vs. AFlow’s $22.50, $0.42 inference cost vs. AFlow’s $1.66, 53 min training wall-clock vs. AFlow’s 184 min and DyLAN’s 508 min, and 19 min inference wall-clock.
- Ablation: removing the textual gradient drops HumanEval from 92.85 to 90.17 and MATH from 51.82 to 48.23; removing early exit raises HumanEval cost from 1.01 to 1.67 and MATH cost from 0.86 to 1.04 (10^-3 $).
Methods
- Search space: agentic operators such as CoT, LLM-Debate, Self-Consistency, Self-Refine, Ensemble, Testing, ReAct, and early exit; a multi-agent system is a DAG, and the agentic supernet is an L-layer probabilistic distribution over operators. A controller network samples query-dependent architectures using MoE-style activation scores with threshold thres = 0.3 and an early-exit operator.
- Optimization: cost-constrained objective with trade-off λ; distribution π is updated via empirical Bayes Monte Carlo gradient with cost-aware importance weights, while operators are updated via agent-based textual gradient for prompts, temperature, and node structure. Settings include L = 4, K = 4, and λ ∈ {1e-3, 5e-3, 1e-2}.
- Evaluation: six public benchmarks—GSM8K, MATH, MultiArith, HumanEval, MBPP, and GAIA—with a 1:4 train:test split; MATH uses 617 selected problems. Base LLM is gpt-4o-mini, with Qwen-2.5-72b/70b and llama-3.1-70b for transfer tests; baselines include CoT, ComplexCoT, Self-Consistency, MultiPersona, LLM-Debate, LLM-Blender, DyLAN, AgentVerse, MacNet, AutoAgents, GPTSwarm, ADAS, AgentSquare, and AFlow.
Limitations
- Stated limitations: Not stated in the paper; the ethical statement says the framework raises no ethical concerns.
- Evident: the operator repository is manually initialized and customizable, so search quality depends on the chosen operators; the textual gradient component is adapted from ADAS and AgentSquare and relies on LLM-generated textual updates.
- Evident: evaluation covers six benchmarks and a limited set of LLM backbones; the text inconsistently says “six” and “seven” benchmarks and lists SVAMP in the introduction but MultiArith in the experiments, which may affect reproducibility and generalization claims.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
24 Own result · 6 Own interpretation · 4 No basis given · 2 Sources not found · 2 No full text of the sources · 1 Partly supported · 1 Not checked yet · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“for instance, in the GAIA benchmark (Mialon et al., 2023), there is no single system that is optimal for both fi le reading and web searching tasks, leaving practitioners with no alternative but to split the benchmark and optimize separately (Zhuge et al., 2024).”
What the sources say
- Partly supportedcites Mialon et al., 2023 sourceExperimental — accuracy not yet measured
In the GPT4 plugins discussion (page unavailable), the paper says evaluators had to choose between a file-reading mode and third-party plugins selected for each task. Figure 5 compares scores by capability, but the text does not establish that no single system is optimal for both file-reading and web-searching tasks. The recommendation to split the benchmark is attributed to Zhuge et al., not this source. (LLM, conf=0.78)
- Could not verifycites Zhuge et al., 2024 sourceExperimental — accuracy not yet measured
Section 3.4 (GAIA; page unavailable) reports overall benchmark accuracy for systems using different tools, including WebSearch and FileAnalyzer. It does not report which system performs best on file-reading tasks versus web-search tasks, so our copy cannot establish that no single system is optimal for both. Searches for that comparison and for separate optimization found no supporting passage. (LLM, conf=0.72)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the more challenging queries (c) and (d), MaAS sampled additional layers”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“MaAS achieves an average best score of 83 . 59% , demonstrating its versatility and superiority.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“removing the textual gradient causes the largest performance drop”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the agentic supernet optimized by MaAS transfers well to models such as Qwen-2.5-70b , with 4 . 98% ∼ 5 . 50% ↑ in performance”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Obs. ❶ MaAS achieves optimal performance across all task domains.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“surpassing existing handcrafted or automated multi-agent systems by 0 . 54% ∼ 16 . 89%”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Obs. ❹ MaAS learns to query-aware early exit from the reasoning process.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“when faced with the easy queries (a) and (b), MaAS exits multi-agent architecture sampling at the second layer with probabilities of 0 . 37 and 0 . 47 , respectively, selecting the early-exit operator.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“❹ inductive , demonstrating strong generalizability to unseen agentic operators.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 14
- The quote only partly states this claim. 1
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“MaAS learns to avoid overly confident early stopping and instead prioritizes testing and self-refinement in deeper layers.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“query (b) chose two agentic operators at the first layer: direct I/O and ReAct”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“DyLAN taking 508 minutes and GPTSwarm taking 129 minutes. In contrast, the optimization wall-clock time of MaAS requires only 53 minutes.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The multi-agent system optimized by MaAS outperforms manually designed methods by an average of 3 . 90 ∼ 6 . 40% and existing automated methods by 2 . 07 ∼ 8 . 26% .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“MaAS achieves the highest accuracy with an API cost of 0 . 42$ , demonstrating its high performance and token economy.”
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Removing O exit and C ( · ) results in little impact on performance, but it weakens MaAS's query-dependent nature and unnecessarily increases the inference cost.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“❸ transferable across datasets and LLM-backbones”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“among the various optimization-oriented agentic workflows, MaAS achieves the highest accuracy with the least training token consumption.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“MaAS has the lowest prompt/completion token consumption, the lowest API cost, and the shortest wall-clock time during inference.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we demonstrate that MaAS is both training/inference cost-efficient from the following three dimensions”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“❷ tokeneconomical , outperforming the SOTA baseline AFlow on the MATH benchmark with 15% of the training cost and 25% of the inference cost”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“MaAS can adaptively sample customized agentic systems for different domains, achieving 18 . 38% and 17 . 61% improvements on Level 1 and 2 tasks, respectively.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We respectfully note that the selection of these operators is highly customizable, allowing users the flexibility to incorporate their desired operators into the operator repository of MaAS.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We introduce the early exit operator, which interrupts the multi-agent architecture sampling process and enables the depth of the agentic supernet to be query-dependent.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“LLM-Debate (Du et al., 2023) allows multiple LLMs to debate, leveraging diverse perspectives to identify better solutions.”
What the sources say
- Not matchedcites Du et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“CoT (Wei et al., 2022) reasoning encourages the LLM to think step by step rather than directly outputting an answer.”
What the sources say
- Not checked — no full text of the sourcecites Wei et al., 2022 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Following (Yao et al., 2023), this operator enables the agent to leverage versatile tools, including code interpreter, web searching, external knowledge database, etc. , to handle diverse user demands.”
What the sources say
- Not matchedcites Yao et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This approach enables fine-grained resource allocation tailored to query difficulty and domain, significantly improving efficiency while maintaining high-quality outputs.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Ph.D.-level abstract algebra may indeed require complicated, token-heavy systems, while simple elementary-level arithmetic works well with a single zero-shot I/O.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
In the paper
“This approach enhances its capability to solve complex problems through intermediate reasoning steps, improving task handling and providing greater transparency in the decision-making process.”
What the sources say
- Not checked — no full text of the sourcecites Wei et al., 2022 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“MaAS introduces a new paradigm in multi-agent system design by replacing static, one-sizefits-all architectures with a dynamic and adaptive agentic supernet.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
The difficulty and domain of a query can be captured by the controller network from the query embedding to select an appropriate multi-agent architecture.
“The core of MaAS lies in tailoring a customized multi-agent system for each user query, which may vary in difficulty and domain, to deliver a satisfactory solution”
The paper relies on this without support.
Token cost adequately represents the inference cost of a multi-agent system.
“C ( · ) evaluates the cost of multi-agent systems, represented by token cost”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“While AFlow's accuracy is comparable to that of MaAS, its training cost reaches 22 . 50$ , which is 6 . 8 × that of MaAS (merely 3 . 38$ ).”
Reported 6.8 · recomputed 6.656805
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| AFlow (Zhang et al., 2024c) | 91 . 16 ↑ 3 . 71 | 51 . 28 ↑ 4 . 91 | 96 . 22 ↓ 0 . 63 | 90 . 93 ↑ 3 . 85 | 81 . 67 ↑ 9 . 84 | 82 . 25 |”
Reported 4.91 · recomputed 4.99
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 1. Introduction · p. 2
“We conduct comprehensive evaluations on seven widely adopted benchmarks, covering diverse use cases in code generation (HumanEval, MBPP), mathematical reasoning (GSM8K, MATH, SVAMP), and diverse tool usage (GAIA).”
§ 4.1. Experiment Setup · p. 5
“Weevaluate MaAS on six public benchmarks covering three domains: (1) math reasoning , GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021), and MultiArith (Roy & Roth, 2016)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.3. Cost Analysis · p. 7
“Table 3 further highlights that MaAS has the lowest prompt/completion token consumption, the lowest API cost, and the shortest wall-clock time during inference.”
§ 4.3. Cost Analysis · p. 7
“| GPTSwarm | 21 , 325 , 266 | 6 , 369 , 884 | 7 . 02$ | 129 | 3 , 105 , 571 | 788 , 273 | 0 . 93$ | 30 | 47 . 88 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 2. Related Work · p. 4
“Similarly, P and T represent prompts and tools. m and n denote the number of LLM-agents and tools invoked in the operator, respectively.”
§ A. Notations · p. 13
“| T | The temperature setting of the LLM. |”
Citations
Cites
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · cited 1×
“S is given by: ) , where v ( · ) denotes the embedding function using lightweight models like MiniLM (Wang et al., 2020) and Sentence-Bert (Reimers, 2019), and ∥ represents concatenation.”
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,705 · cited 1×
“Core NAS techniques, such as reinforcement learning (Zoph, 2016), evolutionary algorithms (Liu et al., 2021), Bayesian optimization (BO) (White et al., 2021), and MCTS (Wang et al., 2021), have inspired analogous approaches in agentic automation, from policy gradient in (Zhuge et al., 2024) to…”
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · cited 4×
“Most existing single/multi-agent workflows can be viewed as agentic operators: CoT (Wei et al., 2022) can be considered one with m = 1 and n = 0 , denoted as O CoT ; Self-RAG (Asai et al., 2023) similarly involves m = 1 agent allocation, but is equipped with n = 1 retrieval engine, denoted as O…”
InfluentialSelf-citationDARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248 · cited 1×
“In NAS, however, these black-box methods were eventually eclipsed by efficient su-pernet training (White et al., 2023), culminating in seminal works like DARTS (Liu et al., 2018) and SNAS (Xie et al., 2018).”
Self-Refine: Iterative Refinement with Self-Feedback
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.17651 · Cited by 2,314 · cited 1×
“Following Madaan et al. (2023), this operator initially generates an answer using CoT reasoning, then prompts the agent to self-reflect iteratively.”
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · cited 4×
“Efforts to automate the design of agent-based systems can be broadly classified into the following categories: (I) Prompt Optimization , such as PromptBreeder (Fernando et al., 2023), DsPy (Khattab et al., 2023), and EvoPrompt (Guo et al., 2023); (II) Inter-agent Communication , which focuses on orchestrating interact…”
InfluentialAutoML: A survey of the state-of-the-art
He, X. J., Zhao, K., & Chu, X. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106622 · Cited by 1,912 · cited 2×
“AutoML.”
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023) · arXiv · doi:10.48550/arxiv.2305.14325 · Cited by 1,202 · cited 12×
“The agentic supernet can be seen as a cascaded multi-layer workflow, including ❶ multiple agentic operators ( e.g. , CoT (Wei et al., 2022), Multi-agent Debate (Du et al., 2023), ReAct (Yao et al., 2023)), as well as ❷ the parameterized probability distributions of operators across layers.”
InfluentialReflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017) · arXiv · doi:10.48550/arxiv.1701.06538 · Cited by 933 · cited 1×
“We implement the sampling process π ϕ with a Mixture-of-Expert (MoE)-style network (Shazeer et al., 2017; Huang et al., 2024): where S ↓ = sort( S , desc ) , and represents the activation scores of all feasible operators w.r.t. q .”
Research works printed but not linked: 1
- Chen, M., Tworek, J., Jun, H., Yuan, Q., Ponde de Oliveira Pinto, H., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C., Tillet, P., Petroski Such, F., Cummings, D., Plappert, M., Chantzis, F., Barnes, E., Herbert-Voss, A., Hebgen Guss, W., Nichol, A., Paino, A., Tezak, N., Tang, J., Babuschkin, I., Balaji, S., Jain, S., Saunders, W., Hesse, C., Carr, A. N., Leike, J., Achiam, J., Misra, V., Morikawa, E., Radford, A., Knight, M., Brundage, M., Murati, M., Mayer, K., Welinder, P., McGrew, B., Amodei, D., McCandlish, S., Sutskever, I., and Zaremba, W. Evaluating large language models trained on code, July 01, 2021 2021.
Cited by
Sorted by citations
Large Language Models Meet NLP: A Survey
Qin, L.-B., Chen, Q., Feng, X., Wu, Y., Zhang, Y., Li, Y.-H., Li, M., Che, W.-X., & Yu, P. S. (2024) · Frontiers Comput. Sci. · doi:10.48550/arxiv.2405.12819 · Cited by 191
AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?Preprint
Zhang, G.-M., Wang, J., Chen, J., Zhou, W., Wang, K., & Yan, S.-C. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2509.03312 · Cited by 145
“1, CRITIC consistently degrades performance ( e.g. , CRITIC+MaAS+GAIA accuracy drops by − 4 .”
Self-citationOn the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents
Huang, J.-T., Zhou, J., Jin, T., Zhou, X., Chen, Z., Wang, W., Yuan, Y., Lyu, M. R., & Sap, M. (2024) · arXiv · doi:10.48550/arxiv.2408.00989 · Cited by 1
G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems
Zhang, G.-M., Fu, M., Wan, G.-C., Yu, M., Wang, K., & Yan, S.-C. (2025) · Neural Information Processing Systems · doi:10.48550/arxiv.2506.07398 · Cited by 124
“More recent automated MASs, such as GPTSwarm [47], ADAS [48], AFlow [49], and MaAS [50] have made it to automatically optimize inter-agent topologies or prompts, which, nevertheless, ultimately yield giant and cumbersome MAS architectures, lacking the agility to self-adjust with accumulated…”
Self-citationMemEvolve: Meta-Evolution of Agent Memory SystemsPreprint
Zhang, G., Ren, H., Zhan, C., Zhou, Z., Wang, J., Zhu, H., Zhou, W., & Yan, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.18746 · Cited by 108
“Storage can be vector databases (Zhao et al., 2024), knowledge graphs (Zhang et al., 2025b; Rasmussen et al., 2025), or others.”
InfluentialSelf-citation
Metadata
- Authors
- Zhang, G., Niu, L., Fang, J., Wang, K., Bai, L., & Wang, X.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2502.04180
- OpenAlex
- W4407247211
- Cited by
- 0
- References
- 99
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- multi-agent systems, architecture search, large language models, tool calls, inference cost reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-24, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-24 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
